Synthetic Personality Review

Investor risk profiles of large language models

Applications, bias, and safety2026arXivApproved editorial review

Authors: Hanyong Cho, Geumil Bae, Jang Ho Kim

Keywords: Persona conditioning, Psychometrics

Source: Open primary source (opens in a new tab)

3
Authors
8
Findings
31
Limitations
5
Evidence

Editorial summary

English

The study administers a seven-item Charles Schwab questionnaire one hundred times to GPT-4o, Gemini 1.5 Pro and Llama 3.1-70B, starting a new conversation for each repetition. Under the generic prompt, GPT has the highest mean risk score and variability, Gemini gives a moderate and constant response, and Llama is the most conservative of the three. The paper then separately adds labels for risk appetite, age, wealth or experience; scores rise with 'risk-seeking', youth, greater wealth and experience. This demonstrates prompt sensitivity, not a latent investor profile: the intervention directly names the desired risk direction, while the other labels overlap questions about horizon, knowledge and owned assets. One hundred runs are samples from a fixed endpoint, not one hundred investors, and dates, providers, settings, exact prompts, outputs, code and data are missing. The tables display 224 p values without correction, contain zero-variance groups and incorrectly describe `nan` as acceptance of the null hypothesis. The evidence comes from a poster at a non-archival workshop and does not evaluate real investors, advisors, portfolios, regulatory suitability or financial harm. It should be read as a case study of questionnaire-response sensitivity to model and persona wording, not as validation of profiling or personalized advice.

Español

El estudio administra cien veces un cuestionario Charles Schwab de siete ítems a GPT-4o, Gemini 1.5 Pro y Llama 3.1-70B, reiniciando la conversación en cada repetición. Bajo el prompt genérico, GPT produce la mayor puntuación media de riesgo y variabilidad, Gemini una respuesta moderada y constante, y Llama la puntuación más conservadora de los tres. Después añade por separado etiquetas de apetito de riesgo, edad, riqueza o experiencia; las puntuaciones aumentan con 'risk-seeking', juventud, mayor riqueza y experiencia. Esto demuestra sensibilidad al prompt, pero no un perfil inversor latente: la intervención nombra directamente el riesgo deseado y las otras etiquetas solapan con preguntas sobre horizonte, conocimiento y activos. Las cien ejecuciones son muestras de un endpoint fijo, no cien inversores, y faltan fechas, proveedores, parámetros, prompts exactos, salidas, código y datos. Las tablas muestran 224 p-valores sin corrección, grupos con varianza cero y `nan` descrito incorrectamente como aceptación de la hipótesis nula. La evidencia procede de un póster de un workshop no archival y no evalúa inversores reales, asesores, carteras, adecuación regulatoria ni daño financiero. Debe leerse como un caso de sensibilidad de respuestas de cuestionario a modelo y persona, no como validación de perfilado o asesoramiento personalizado.

Research question

Do the response distributions of GPT-4o, Gemini 1.5 Pro and Llama 3.1-70B differ on an investor profile questionnaire, and do they change when the prompt specifies risk appetite, age, wealth or experience?

Method

Seven multiple-choice questions from the Charles Schwab investor profile questionnaire are formulated sequentially: two generate a horizon score and five a risk tolerance score. Each questionnaire is repeated one hundred times per model and condition, with a new conversation. In addition to the generic prompt, a single attribute is varied: three risk levels, four age decades, three qualitative wealth levels and three experience levels. Means and most frequent profiles are compared using ANOVA and Kruskal-Wallis.

Sample: One hundred new conversations per model and condition. The design involves 4,200 questionnaires: 300 under the generic prompt and 3,900 under thirteen persona conditions. The observed unit is a stochastic generation of a fixed endpoint; there are no persons, advisors, independently trained models or temporal waves of replication.

Findings

  • The default horizon/risk means are 16.86/24.68 for GPT, 18.00/20.00 for Gemini and 17.69/19.68 for Llama.
  • Gemini produces the same default profile across the one hundred repetitions, Llama varies somewhat and GPT shows the greatest dispersion under unpublished parameters.
  • The risk prompt moves the tolerance mean from approximately 9-11 in risk-averse to 34-39 in risk-seeking across the three models.
  • The 20s condition obtains the highest risk score and the 50s the lowest across the three models, although Llama is not monotonic between 30s and 40s.
  • Risk scores increase with qualitative wealth labels across the three endpoints.
  • The absence of experience strongly reduces the score relative to partial or professional experience.
  • The results support that model, wording and persona alter responses; they do not distinguish internalization from semantic compliance.
  • The article itself recommends caution regarding the use of LLMs in personalized financial advice.

Limitations

  • The models receive the instruction to assume they are investors; the outputs are role-play and not preferences possessed by the system.
  • The assumed default profile depends on a single wording, order, endpoint and protocol and is not tested as a stable trait.
  • Risk-averse, risk-neutral and risk-seeking directly specify the direction that the questionnaire then measures.
  • Age, wealth and experience semantically overlap with items on horizon, knowledge and assets, so the effect may be content echo.
  • Only one attribute is modified at a time; there are no intersectional profiles, conflicts between traits or interactions.
  • Lower/middle/upper wealth and partial/professional experience lack monetary, geographic or behavioral definitions.
  • Age is limited to 20s-50s and excludes older investors.
  • A single commercial seven-item questionnaire is used, with no alternative forms, reordering, paraphrasing or psychometric validation for model outputs.
  • The 2024 snapshot and its exact score are not archived; the current Schwab URL loads dynamically.
  • There is no human sample, authorized advisors, known profiles, observed behavior or external criterion of suitability.
  • Portfolio recommendations, returns, loss, explanation, constraints or correspondence between score and product are not evaluated.
  • One hundred completions from one endpoint measure its stochastic distribution, not one hundred investors or uncertainty across trained models.
  • Temperature, top-p, seeds, system messages, limits, invalid response handling, retries and failure counts are missing.
  • Provider, API, date, region or runtime are not published; Gemini is a mutable alias and Llama does not specify checkpoint, quantization or server.
  • Only the seven questions are summarized; the exact per-turn messages and the scoring implementation are not published.
  • Responses, conversations, per-questionnaire scores, code, environment, logs or analysis files are not published.
  • The 4,200 questionnaires and 29,400 implicit turns and possible missing data are not explicitly reconciled.
  • ANOVA is reported despite ordinal data, minimal/zero variance and rejection of homogeneity; Levene's test is not shown.
  • Kruskal-Wallis does not generally test equality of medians without additional assumptions, despite that description in the text.
  • The note to Table 9 interprets `nan` as acceptance of the null hypothesis; an undefined statistic does not allow accepting H0.
  • Gemini-risk-Q7 shows ANOVA `nan` and Kruskal-Wallis 0.0000, contradicting a uniform reading of `nan` as no difference.
  • Tables 4 and 9 contain 56 and 168 p-values, 224 in total, with no correction for multiplicity or confirmatory hierarchy.
  • The 0.0000 values are roundings, not exactly zero probabilities, and no thresholds or greater precision are given.
  • Effect sizes, confidence intervals, mean uncertainty and post-hoc procedures are missing.
  • The tests generalize, at most, to executions of the fixed endpoint and configuration, not to families, versions, providers or times.
  • Financial inference confuses risk preference with capacity to bear losses, although the introduction distinguishes both concepts.
  • Age, wealth and experience are insufficient for suitability: objectives, income, debts, liquidity, dependents, taxation and jurisdiction are missing.
  • The patterns may reinforce simple stereotypes (young, rich or expert implies more risk) without studying equity, exceptions or harm.
  • Calibration, false risky/conservative profiles, contradictory prompts, injection, abstention or human scaling are not evaluated.
  • The presentation was a poster at a non-archival workshop with no official proceedings, not an ACM conference paper.
  • There is no data/code availability section, identifiable author repository or supplementary appendix.

What the study does not establish

  • It does not establish stable investor personalities or preferences within the models.
  • It does not demonstrate correct inference of an undeclared human profile; the attributes are delivered explicitly.
  • It does not separate genuine personalization from obedience to labels and echo between prompt and items.
  • It does not psychometrically validate the questionnaire for LLMs nor compare profiles with persons or experts.
  • It does not demonstrate financial capacity, regulatory suitability or quality of a portfolio recommendation.
  • It does not test safety, benefit, calibration or absence of harm in real advice.
  • It does not generalize across prompts, order, languages, instruments, checkpoints, providers or dates.
  • It does not convert one hundred samples from an API into one hundred investors or independent replicas of a model.
  • It does not allow reproducing scores and p-values without outputs, prompts, parameters, code and analysis files.
  • It does not justify that young, rich or expert persons should automatically receive riskier recommendations.
  • It does not constitute an archival publication in ICAIF proceedings.

Traceability

Scope: Full text

Version: arXiv:2603.09303v2; poster at AI for Finance Symposium '25 (non-archival)

Consulted source: https://arxiv.org/abs/2603.09303

Review: Codex 6-page visual full-text, prompt-construct, sampling, statistical, financial-suitability, data/code and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o, snapshot gpt-4o-2024-08-06
  • Gemini 1.5 Pro, alias gemini-1.5-pro
  • Llama 3.1 70B, identificador llama3.1-70b

Instruments and metrics

  • Cuestionario Charles Schwab de perfil inversor, versión 2024 según los autores
  • Dos ítems y puntuación de horizonte temporal
  • Cinco ítems y puntuación de tolerancia al riesgo
  • Prompts de apetito de riesgo, edad, riqueza y experiencia
  • ANOVA y prueba de Levene mencionada pero no publicada
  • Kruskal-Wallis sobre respuestas ordinales

Data used

  • 300 cuestionarios por defecto, 100 por cada uno de tres endpoints, no publicados
  • 3.900 cuestionarios condicionados por trece valores de persona y tres endpoints, inferidos del diseño pero no reconciliados en el artículo
  • Hasta 29.400 respuestas individuales implícitas en 4.200 cuestionarios de siete ítems, sin publicación ni registro de fallos
  • Snapshot del cuestionario Schwab 2024 no archivado con los materiales

Evidence and location

  • Method, prompts, instrument, tables, figures, results, conclusion and references: arXiv:2603.09303v2, all 6/6 PDF pages rendered and individually inspected
  • Versions, dates, license and presentation comment: Official arXiv abstract and Atom metadata inspected 2026-07-17
  • Poster condition and non-archival character without proceedings: Official AI for Finance Symposium '25 event and accepted-papers page inspected 2026-07-17
  • Absence of identifiable repository and reproduction materials: Official arXiv e-print PDF plus authenticated GitHub repository/code searches inspected 2026-07-17
  • Construct audit, prompt echo, inferential unit, statistics, financial suitability and reproducibility: reports/verification/article-394-investor-risk-profile-prompt-echoing-pseudoreplication-statistics-financial-suitability-data-code-and-reproducibility-audit.json