Synthetic Personality Review

How Personality Traits Influence Negotiation Outcomes? A Simulation based on Large Language Models

Society, culture, and collective behavior2024ACL AnthologyApproved editorial review

Authors: Yin Jou Huang, Rafik Hadfi

Keywords: Personality traits, Large language models, Negotiation simulation, Decision-making, Big Five, Bargaining dialogues

Source: Open primary source (opens in a new tab)

2
Authors
13
Findings
46
Limitations
14
Evidence

Editorial summary

English

The paper does not measure the personality of human negotiators or train an agent on observed psychological traits. It studies what happens when two LLMs, a buyer and a seller, are assigned synthetic Big Five profiles through adjectives in their prompts. Each dimension takes one of six levels, positive or negative polarity at low, moderate, or high degree, and contributes three adjectives; the fifteen adjectives are shuffled and appended to economic buyer or seller instructions. The main experiment uses GPT-4 0613, 60 stated CraigsListBargain products, a maximum of 20 turns, and 1,499 dialogues. A third GPT-4 0613 classifies each utterance as offer, pondering, acceptance, breakdown, or chit-chat, extracts the price, and produces a free-text strategy label. Outcomes are related to own utility, joint utility, concession, success, and duration through Spearman correlations.

For GPT-4, every significant association in the main table has |ρ| < 0.30. Seller Agreeableness shows the most consistent pattern: it is associated with lower own utility (ρ = −0.262), but higher joint utility (0.118), more concession (0.261), only a slightly higher agreement probability (0.052), and fewer turns (−0.223). Seller Conscientiousness is associated with higher own utility (0.127) and less concession (−0.097), while buyer Conscientiousness is related to somewhat longer negotiations (0.083). Buyer Extraversion is weakly associated with success (0.072), and seller Extraversion with joint utility (0.069). Neuroticism and Openness have still smaller effects. These results support the claim that personality wording in a prompt slightly changes simulated negotiation trajectories. They do not show large psychological effects or reliable individual profiles.

The strategy analysis depends entirely on labels generated by the evaluator LLM and grouped manually. Figure 2 associates accommodating (β = 0.125) and concession (0.093) with higher joint utility, and assertiveness with lower joint utility (−0.261). Appendix D, however, assigns assertiveness a joint coefficient of −0.45, which does not match the figure. The qualitative examples contain offers outside the supplied bounds and invented emotional appeals, but calling them “deception” or “strategy” interprets textual behavior without validating intent.

The human comparison is shallow. Original CraigsListBargain dialogues average 8.47 turns and 108.82 words, versus 7.07 turns and 272.17 words in the simulation. The paper also reports positive correlations between price and length, r = 0.194 for turns and 0.242 for words. It does not compare people and agents with matched Big Five profiles, analyze equivalent human outcomes, or run a behavioral equivalence test. Similarity consists of selected aggregate patterns and agreement with prior literature rather than a direct replication of human negotiation.

The appendices add GPT-3.5 Turbo 0125 and Meta-Llama-3-70B-Instruct. Llama reproduces several GPT-4 directions. The textual claim that GPT-3.5 yields “no significant correlations” contradicts Table 4, which marks eight relationships at p < 0.05 and two more at p < 0.10; the stated model comparison therefore cannot be accepted as written. In an IPIP-50 check with 300 reported agents, diagonal correlations between instructed level and questionnaire response are 0.73–0.78 for GPT-4, 0.59–0.84 for GPT-3.5, and 0.85–0.92 for Llama. This check demonstrates instruction following and internal coherence within the same verbal frame, not personality validity: the model is explicitly given the adjectives and answers related public items, with demand effects and contamination risk acknowledged by the authors.

Reproducibility is inadequate. The official repository has one commit and releases scripts, a strategy list, and 161 negotiation entries, but not the 1,499 dialogues, exact profiles, outputs, the selected 60 products, IPIP items, or executed commands. It sets no seeds, and generation uses temperature 1. The CSV would produce 314 usable configurations by default rather than the reported experiment. It also pins openai==0.27.8 while calling the 1.x API and imports pandas in eval.py without declaring it, so the published environment cannot run the workflow without repair. The defensible contribution is an exploratory framework showing that Big Five instructions can weakly modulate negotiations between specific LLM snapshots. It does not establish human personality, real negotiation behavior, intent, human parity, or a validated basis for deploying negotiation agents.

Español

El artículo no mide la personalidad de negociadores humanos ni entrena un agente con rasgos psicológicos observados. Estudia qué ocurre cuando se asignan a dos LLM, comprador y vendedor, perfiles Big Five sintéticos mediante adjetivos incluidos en el prompt. Cada dimensión adopta uno de seis niveles, polaridad positiva o negativa y grado bajo, moderado o alto, y aporta tres adjetivos; los quince se mezclan y se añaden a instrucciones económicas de compra o venta. El experimento principal usa GPT-4 0613, 60 productos declarados de CraigsListBargain, un máximo de 20 turnos y 1.499 diálogos. Un tercer GPT-4 0613 clasifica cada intervención como oferta, reflexión, aceptación, ruptura o charla, extrae el precio y genera una etiqueta libre de estrategia. Los resultados se relacionan con utilidad propia, utilidad conjunta, concesión, éxito y duración mediante correlaciones de Spearman.

En GPT-4, todas las asociaciones significativas de la tabla principal tienen |ρ| < 0,30. La amabilidad del vendedor muestra el patrón más consistente: se asocia con menor utilidad propia (ρ = −0,262), pero mayor utilidad conjunta (0,118), mayor concesión (0,261), una probabilidad de acuerdo apenas superior (0,052) y menos turnos (−0,223). La responsabilidad del vendedor se asocia con más utilidad propia (0,127) y menos concesión (−0,097), mientras la responsabilidad del comprador se relaciona con negociaciones algo más largas (0,083). La extraversión del comprador se asocia débilmente con éxito (0,072) y la del vendedor con utilidad conjunta (0,069). Neuroticismo y apertura presentan efectos todavía menores. Estos datos apoyan que modificar el prompt de personalidad altera ligeramente las trayectorias de negociación simuladas; no muestran efectos psicológicos grandes ni perfiles individuales fiables.

El análisis de estrategias depende enteramente de etiquetas generadas por el LLM evaluador y agrupadas manualmente. La Figura 2 relaciona acomodación (β = 0,125) y concesión (0,093) con más utilidad conjunta, y asertividad con menos (−0,261). Sin embargo, el Apéndice D atribuye a la asertividad un coeficiente conjunto de −0,45, cifra que no coincide con la figura. Los ejemplos cualitativos muestran ofertas fuera de los límites dados y apelaciones emocionales inventadas, pero denominarlas «engaño» o «estrategia» interpreta conducta textual sin validar intención.

La comparación humana es superficial. Los diálogos originales de CraigsListBargain promedian 8,47 turnos y 108,82 palabras, frente a 7,07 turnos y 272,17 palabras en la simulación. El estudio también informa correlaciones positivas entre precio y longitud, r = 0,194 para turnos y 0,242 para palabras. No compara personas y agentes con los mismos perfiles Big Five, no analiza resultados humanos equivalentes ni realiza una prueba de equivalencia conductual. La semejanza consiste en algunos patrones agregados y coincidencias con literatura previa, no en una réplica directa de negociación humana.

Los apéndices añaden GPT-3.5 Turbo 0125 y Meta-Llama-3-70B-Instruct. Llama reproduce varias direcciones de GPT-4. La afirmación textual de que con GPT-3.5 «no se puede identificar ninguna correlación significativa» contradice su Tabla 4, que marca ocho relaciones con p < 0,05 y otras dos con p < 0,10; por ello no puede sostenerse esa comparación de modelos tal como está escrita. En un control IPIP-50 con 300 agentes declarados, las correlaciones diagonales entre nivel instruido y respuesta al cuestionario son 0,73–0,78 para GPT-4, 0,59–0,84 para GPT-3.5 y 0,85–0,92 para Llama. Ese control muestra obediencia y coherencia dentro del mismo marco verbal, no validez de personalidad: el modelo recibe explícitamente los adjetivos y contesta ítems públicos relacionados, con riesgo de demanda y contaminación reconocido por los autores.

La reproducibilidad es insuficiente. El repositorio oficial solo tiene un commit y publica scripts, una lista de estrategias y 161 entradas de negociación, pero no los 1.499 diálogos, los perfiles exactos, los resultados, el subconjunto de 60 productos, los ítems IPIP ni las órdenes utilizadas. No fija semillas; el generador usa temperatura 1. El CSV produciría por defecto 314 configuraciones utilizables, no el experimento descrito. Además, fija openai==0.27.8 mientras llama a la API de la serie 1.x y eval.py importa pandas sin declararlo, por lo que el entorno publicado no ejecuta el flujo sin reparación. La contribución defendible es un marco exploratorio que muestra que instrucciones Big Five pueden modular débilmente negociaciones entre snapshots concretos de LLM. No demuestra personalidad humana, negociación real, intención, paridad con personas ni una base validada para desplegar agentes negociadores.

Research question

How do the outcomes and strategies of simulated bilateral negotiations change when synthetic Big Five profiles are assigned to buyer and seller LLM agents, and to what extent do those patterns resemble trends described in human negotiations?

Method

Exploratory simulation of negotiation. Buyer and seller receive opposing objectives and random Big Five profiles with six levels per dimension; each level is expressed through three Goldberg adjectives and intensity modifiers, fifteen adjectives per agent in mixed order. The main analysis generates 1,499 declared dialogues over 60 CraigsListBargain entries with GPT-4 0613 and a declared limit of 20 turns. Another GPT-4 0613, via function calling, detects state, offer, and strategy. Own and joint utility, concession, success, and duration are computed; they are correlated separately with each trait and role. Frequent strategies are manually clustered, descriptive linear regressions are fitted, and aggregate length is compared with human dialogues. The appendices repeat associations with GPT-3.5 Turbo 0125 and Llama-3-70B-Instruct and administer IPIP-50 to 300 declared agents. The editorial audit read and rendered the 16 pages and reviewed the single commit of the official repository.

Sample: The main experiment declares 1,499 GPT-4 dialogues built from 60 CraigsListBargain entries. Each dialogue pits two agents with random profiles of five dimensions and six levels per dimension. The article does not report how many dialogues each product contributes, the distribution of profiles, losses due to error, the overall success rate, or effective sample sizes for each metric. The appendices do not specify the N of GPT-3.5 and Llama dialogues. For IPIP-50, 300 agents are declared, without precisely clarifying the distribution by model, trait, and level.

Findings

  • In the main GPT-4 table, all significant correlations have a magnitude below 0.30, so the observed effects are small.
  • Seller agreeableness is associated with lower own utility (ρ = −0.262), but higher joint utility (0.118), concession (0.261), success (0.052), and fewer turns (−0.223).
  • Seller conscientiousness is associated with more own utility (ρ = 0.127) and less concession (−0.097); buyer conscientiousness is associated with more turns (0.083).
  • Buyer extraversion is weakly related to success (ρ = 0.072) and seller extraversion to joint utility (0.069).
  • Neuroticism and openness show significant associations of magnitude below 0.10 in the main analysis, consistent with a minor role within this simulation.
  • Figure 2 attributes positive joint utility coefficients to accommodation (β = 0.125) and concession (0.093), and a negative one to assertiveness (−0.261), although Appendix D gives another figure for the latter.
  • Human dialogues average 8.47 turns and 108.82 words; simulated ones 7.07 turns and 272.17 words, that is, fewer exchanges but much more verbosity.
  • The article reports r = 0.194 between price and number of turns and r = 0.242 between price and words, positive but small associations.
  • The two qualitative cases exhibit an offer above the listed price, an invented appeal to an elderly relative, and ultimatum behavior; they are selected examples, not population frequencies.
  • The diagonal correlations between instructed level and IPIP are 0.73–0.78 for GPT-4, 0.59–0.84 for GPT-3.5, and 0.85–0.92 for Llama-3-70B.
  • Llama-3-70B reproduces several directions of GPT-4, especially the exchange between agreeableness, own utility, concession, success, and duration.
  • Table 4 of GPT-3.5 contains eight associations marked with p < 0.05 and two with p < 0.10, in contradiction with the phrase in the appendix stating it finds none significant.
  • The repository audit confirms that part of the framework and product metadata are published, but not the dialogues, profiles, outputs, or exact parameters needed to reconstruct the tables.

Limitations

  • The study manipulates instructions of artificial agents; it does not measure personality traits or decisions of people.
  • It does not include a condition without personality instructions, so it does not estimate the absolute change relative to a neutral agent.
  • The five dimensions vary simultaneously and are mainly analyzed with univariate correlations; the effects of one trait are not isolated while holding the others constant.
  • The profile space has 6^5 combinations per agent and is enormous relative to 1,499 dialogues; no counts, balance, or coverage of combinations are published.
  • Sixty products are reused for 1,499 dialogues, but the tests treat the dialogues as independent observations and do not cluster errors by product or configuration.
  • No mixed models or controls by category, price, difference between target prices, description length, or product repetition are applied.
  • Each negotiation results table contains 50 correlations and three models are presented, in addition to 75 IPIP correlations, with no explicit correction for multiple comparisons.
  • p < 0.10 is used as a highlighted category and no confidence intervals are published, which increases the risk of overinterpreting weak findings.
  • The significance of very small coefficients is driven by the nominal N and does not imply practical importance.
  • The effective N of successful negotiations, valid offers, or observations used in each correlation is not reported.
  • The sample sizes of GPT-3.5 and Llama-3-70B are not published, preventing comparison of power and stability between models.
  • There is no preregistration, complete directional hypotheses, or clear separation between confirmatory and exploratory analyses.
  • The simulation covers a single competitive buyer-seller game in English and not other types of negotiation, cultures, languages, or long-term relationships.
  • The economic objectives are linear and omit risk, time, reputation, real incomplete information, alternatives, breakdown costs, and non-monetary preferences.
  • The assumption that the zone of agreement occupies 70 % of the interval between prices is arbitrary and is not subjected to sensitivity analysis.
  • Although utilities are defined with codomain [0,1], offers can fall outside the intervals. The example of listed price 50, target 30, and agreement 60 produces seller utility above 1 and buyer utility negative.
  • Joint utility can be negative outside the zone of agreement, so its interpretation as equity or mutual gain ceases to be valid in cases that the study itself retains.
  • Concession is derived from a proposed functional form and is not validated against human annotations or a standard measure in these dialogues.
  • The code modifies offer extremes for concession after calculating part of the limits, an implementation that does not cleanly match the published equation and may generate unstable values.
  • An unvalidated GPT-4 0613 determines acceptance, breakdown, price, and strategy; any error by the evaluator alters success, utility, duration, and strategic analysis.
  • The same GPT-4 snapshot generates and evaluates the main experiment, creating shared dependence between behavior and measurement.
  • GPT-4 also evaluates the GPT-3.5 and Llama dialogues, so differences between models may include differential compatibility with the evaluator.
  • Strategies are generated as free text, normalized with a manual list of synonyms, and unmapped labels are omitted; coverage, inter-annotator agreement, and validation are not published.
  • Only strategies with more than 20 appearances are included in the figures, a post-data selection that may hide rare or unstable patterns.
  • The strategic regression treats multiple interventions from the same dialogue as independent and assigns the final utility to all of them, incurring in pseudoreplication and weak temporal attribution.
  • The code uses all strategy dummies with an intercept, a collinear parameterization; the coefficients have no clear reference category and are published without standard errors or tests.
  • The chi-square test counts multiple interventions and ten polarity indicators per dialogue as if they were independent observations, violating the independence of the test.
  • Figure 2 shows −0.261 for assertiveness and joint utility, while Appendix D states −0.45; the discrepancy is not explained.
  • The GPT-3.5 section states that no significant correlations exist, but Table 4 marks eight with p < 0.05 and two with p < 0.10.
  • The buyer and seller receive asymmetric prompts: the seller sees the product description and the buyer does not, which may influence differences attributed to role.
  • Adjectives such as immoral, dishonest, intelligent, or socially progressive include moral valuation, ability, or ideology and may directly change strategy without representing a latent trait.
  • The human comparison does not match profiles, conditions, instructions, or outcomes; it only contrasts aggregate length statistics and previous references.
  • No intervals or tests are published for the differences of 8.47 versus 7.07 turns and 108.82 versus 272.17 words.
  • It is not clear whether the price-length correlations of 0.194 and 0.242 correspond to humans, LLMs, both combined, or averages by category; no p-values or intervals are given either.
  • The code calls intervention positions "round" and includes two preset openings; with the default value of ten cycles it can produce 22 entries, in tension with the declared maximum of 20 turns.
  • The "deception" examples lack verification of beliefs or intention; an invented offer may be hallucination or acting induced by the prompt.
  • The IPIP control again shows the model the adjectives and asks about semantically related items, so it has strong demand and circularity effects.
  • IPIP-50 is public and the authors acknowledge possible contamination; correlation does not demonstrate that the model possesses the psychological construct.
  • The article declares 300 agents for IPIP, but does not clarify the N per model, level, and table, nor does it publish individual responses.
  • The GPT-4 0613 and GPT-3.5 0125 snapshots are retired and responses at temperature 1 cannot be identically regenerated without preserved outputs.
  • The repository does not contain the 1,499 dialogues, generated profiles, IPIP outputs, subset of 60 products, intermediate tables, or reproducible figures.
  • negotiation_settings.csv contains 161 parsed entries, 157 usable by default; there is no script or seed that selects the 60 declared ones and the generator would produce 314 configurations, not 1,499 dialogues.
  • No seeds are set for profiles, adjectives, or selection, and the orders or repetition values that originated the experiment are not documented.
  • The environment sets openai==0.27.8, but the code calls openai.chat.completions and OpenAI, interfaces from the 1.x series; furthermore eval.py requires pandas, absent from the Pipfile and lock.
  • The repository does not include tests, a license, a release, an experimental history, or a commit cited within the paper; only a single initial commit exists.
  • Ethical risks are discussed, but there are no users, harm study, privacy, security, or deployment to validate mitigations.

What the study does not establish

  • It does not demonstrate how human personality causes negotiation outcomes.
  • It does not demonstrate that GPT-4, GPT-3.5, or Llama possess stable, internal Big Five traits comparable to those of a person.
  • It does not validate the synthetic profiles as psychometric measurements; it shows responses consistent with explicit adjectives.
  • It does not demonstrate that the simulations globally reproduce human negotiations; only some selected aggregate patterns coincide.
  • It does not demonstrate that GPT-4 models personality better than GPT-3.5, because the claim about the absence of significance in GPT-3.5 contradicts the table and comparable N are missing.
  • It does not establish that the Llama results generalize to other open models, sizes, versions, or decoders.
  • It does not isolate the causal effect of each trait relative to the other four, the product, the price, and the wording of the adjectives.
  • It does not test that the automatic strategy, success, or price labels are correct.
  • It does not demonstrate that accommodation, concession, or assertiveness cause final utility; the regressions are descriptive and do not respect temporal dependence.
  • It does not demonstrate deception, intention, emotion, economic understanding, or psychological reasoning in the agents.
  • It does not demonstrate that higher joint utility represents fairness, welfare, or a desirable outcome outside of the chosen arbitrary functions.
  • It does not predict performance, safety, or benefit of negotiating agents in real markets, employment, banking, or finance.
  • It does not justify personalizing assistants with psychological traits or using these profiles to influence, persuade, or decide for users.
  • It does not generalize to current models, other languages, multiparty negotiation, repeated relationships, or high-stakes situations.
  • It does not offer a complete independent reproduction of the published figures with the available artifacts.

Traceability

Scope: Full text

Version: Findings of ACL: EMNLP 2024, pp. 10336–10351; DOI 10.18653/v1/2024.findings-emnlp.605; ACL Anthology ID 2024.findings-emnlp.605; CC BY 4.0

Consulted source: https://aclanthology.org/2024.findings-emnlp.605.pdf

Review: Codex full-text, visual, appendix, code and reproducibility audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4 (gpt-4-0613) as buyer and seller in the main experiment
  • GPT-4 (gpt-4-0613) as dialogue state, price and strategy detector in every experiment
  • GPT-3.5 Turbo (gpt-3.5-turbo-0125) in Appendix B
  • Meta-Llama-3-70B-Instruct in Appendix B

Instruments and metrics

  • Synthetic six-level Big Five profiles from Goldberg bipolar adjective pairs
  • IPIP-50 personality questionnaire
  • Spearman rank correlations
  • Pearson correlations for product price and dialogue length
  • Own and Nash-inspired joint utility functions
  • Estimated concession rate
  • Negotiation success rate and number of turns
  • GPT-4 function-calling dialogue state detector
  • Manually consolidated strategy taxonomy
  • One-hot linear regression for strategy coefficients
  • Pearson chi-square residuals for strategy and trait polarity

Data used

  • CraigsListBargain bargaining dialogues and product metadata
  • Sixty negotiation entries reported in the paper
  • 1,499 generated negotiation dialogues reported in the paper but not released
  • Official big5-llm-negotiator repository at commit dce8cb231bf99a596aefb6e415165518e03706e4
  • Repository negotiation_settings.csv with 161 parsed entries, 157 usable by the default generator
  • Repository strategy_list.json with 11 manually consolidated strategy categories

Evidence and location

  • Metadata, abstract, DOI, pages, and license: ACL Anthology 2024.findings-emnlp.605; final paper p. 10336; CC BY 4.0
  • Framework, Big Five profiles, six levels, and fifteen adjectives: Final paper, sections 3.1–3.2 and Table 1, pp. 10338–10339
  • Sixty entries, 1,499 dialogues, maximum of 20, and GPT-4 detector: Final paper, section 4.1, p. 10340
  • Utility functions, concession, success, and duration: Final paper, section 4.2, equations 3–8, pp. 10340–10341
  • GPT-4 correlations by trait and role: Final paper, Table 2 and section 5.1, pp. 10341–10342
  • Strategies, regression, and joint coefficients: Final paper, Figure 2 and section 5.2, pp. 10342–10343; Appendix D and Figure 5, pp. 10349–10351
  • Comparison of human and simulated length and relationship with price: Final paper, section 5.3 and Figure 3, p. 10343
  • Cases of out-of-range offer and emotional appeal: Final paper, Table 3 and section 5.4, p. 10344
  • Limitations and declared ethical risks: Final paper, Limitations and Ethics Statement, p. 10345
  • GPT-3.5, Llama, and significance contradiction: Final paper, Appendix B and Tables 4–5, pp. 10348–10349
  • IPIP-50, 300 agents, and correlations by model: Final paper, Appendix C, Figure 4 and Tables 6–8, pp. 10348–10350
  • Scripts, dependencies, prompts, and evaluation logic: Official big5-llm-negotiator repository commit dce8cb231bf99a596aefb6e415165518e03706e4, Agent.py, Game.py, eval.py, Pipfile and Pipfile.lock, audited 15 Jul 2026
  • Discrepancy between published corpus and declared design: Official repository commit dce8cb2, data/negotiation_settings.csv and agent_profile_generation.py: 161 parsed entries, 157 usable, two default profiles per product; compared with final paper p. 10340
  • Absence of outputs and repository status: Official repository commit dce8cb2: one initial commit dated 2 Nov 2024, no raw dialogues, generated profiles, IPIP data, tests, license or releases; verified 15 Jul 2026