AI can outperform humans in predicting correlations between personality items

Evaluation and psychometric validity2025Nature / Scientific ReportsApproved editorial review

Authors: Philipp Schoenegger, Spencer Greenberg, Alexander Grishin, Joshua Lewis, Lucius Caviola

Keywords: Personality Items, Correlation Forecasting, Large Language Models, Psychometrics, Wisdom of Crowds

Source: Open primary source (opens in a new tab)

5
Authors
18
Findings
24
Limitations
8
Evidence

Editorial summary

English

Schoenegger and coauthors study a narrow but useful task: estimating the empirical correlation between two personality items without collecting new questionnaire responses. The benchmark contains 249 pairs drawn from 103 SAPA items. It is not a natural sample of correlations: one third was selected below −0.2, one third between −0.2 and 0.2, and one third above 0.2. The study compares 254 laypeople, 272 psychology or behavioral-science academics, three temperature-zero GPT-4o runs, three Claude 3 Opus runs, and one deterministic prediction from PersonalityMap, a proprietary network trained specifically on item pairs. Each human judged 30 randomly assigned pairs, apart from some incomplete lay responses, whereas every AI system covered all 249.

In individual comparisons, the seeded randomly selected GPT-4o run had mean absolute error 0.14, Claude 0.11, and PersonalityMap 0.07, compared with participant-level means of 0.29 for laypeople and 0.20 for experts. GPT-4o ranked at the 95.67th percentile against laypeople and 70.22nd against experts; Claude ranked at 100 and 95.22; PersonalityMap ranked at 100 against both. Win rates over the items actually answered by each person were 90.94%/69.85% for GPT-4o, 97.64%/86.40% for Claude, and 100%/99.26% for PersonalityMap against laypeople/experts. This supports the claim that one query to these systems is usually more accurate than one individual on this task. It does not establish broad personality understanding: the target is an item-level population parameter, and the individual comparison gives each AI 249 pairs while each human sees only 30.

Median aggregation by item changes the comparison. Mean error is 0.164 for laypeople, 0.086 for experts, 0.140 for GPT-4o, 0.104 for Claude, and 0.072 for PersonalityMap. Correlations between aggregate predictions and empirical values are 0.88, 0.90, 0.78, 0.80, and 0.91, respectively. The three-bucket test (<−0.1, −0.1 to 0.1, >0.1) reports no group difference, χ²(4)=6.42, p=.170. The published error tests, however, treat five measurements on the same 249 pairs as independent: they use Kruskal–Wallis and label Paired=False Mann–Whitney comparisons as “Dunn’s test.” An editorial paired robustness check yields Friedman χ²(4)=96.59, p<.001 and Holm-adjusted Wilcoxon tests. It preserves the broad ranking but changes two borderline conclusions, GPT-4o versus laypeople becomes p=.047 and PersonalityMap versus experts p=.047, while experts versus Claude becomes p=.053. A paired Cochran Q test still finds no bucket difference, Q(4)=8.70, p=.069. These checks are not a substitute for a formal statistical reanalysis, but they show that some inferential detail depends on ignoring the paired design.

Analyses added after first-round peer review test 30 unseeded GPT-4o queries at temperature 1 and SurveyBot3000. Aggregating 30 higher-temperature runs raises GPT-4o’s prediction–truth correlation to 0.88 without uniformly improving the other metrics. SurveyBot3000 appears competitive, but it was trained on SAPA and therefore on the benchmark used here; the authors correctly state that its result cannot distinguish generalization from memorization. The transparent review record also shows that reviewers pressed the authors to narrow the title from “understanding personality” to predicting item correlations and to curb speculative applications.

The OSF artifact reproduces the headline numbers but not the study end to end. It contains two processed Excel files, two monolithic scripts, and two truth JSON files, with no README, requirements file, lockfile, CLI, or code for collecting the LLM outputs or running PersonalityMap. Paths remain as XXX/...; after reconstructing them and using the package versions listed in the reporting summary, the main script completes and reproduces the published tables. The additional script also completes, but sets num_bootstraps = 10 although the paper reports 10,000, produces intervals different from the supplement, and returns nan CIs for two inapplicable conditions. Both scripts print the Shapiro p-value as both W and p, and the Levene p-value as both F and p. PersonalityMap remains proprietary: two authors created it, and one also founded Positly and GuidedTrack, all disclosed conflicts; the public artifact cannot independently verify that none of the evaluated items entered its training data.

The defensible contribution is that synthetic item-correlation forecasts can be cheap and surprisingly accurate, while aggregated human expertise remains competitive with general LLMs. The proper scope is linear, cross-sectional self-report correlations from one English inventory under an artificially stratified test distribution. The evidence does not validate hiring, diagnosis, marketing, automated psychological assessment, or wholesale replacement of human data. Nor does a predicted correlation establish causation: the authors themselves state that scientific uses must ultimately be confirmed in real humans and that PersonalityMap still lacks prediction-level uncertainty estimates.

Español

Schoenegger y coautores estudian una tarea estrecha pero útil: estimar la correlación empírica entre dos ítems de personalidad sin recoger nuevas respuestas al cuestionario. El conjunto contiene 249 pares de 103 ítems de SAPA. No es una muestra natural de correlaciones: se construyó con un tercio por debajo de −0,2, un tercio entre −0,2 y 0,2 y un tercio por encima de 0,2. Comparan 254 personas no expertas, 272 académicos de psicología o ciencias del comportamiento, tres ejecuciones de GPT-4o a temperatura 0, tres de Claude 3 Opus, y una predicción determinista de PersonalityMap, una red propietaria entrenada específicamente con pares de ítems. Cada persona contestó 30 pares asignados al azar, salvo algunos legos incompletos, mientras cada sistema cubrió los 249.

En las comparaciones individuales, una ejecución seleccionada con seed fijo de GPT-4o tuvo error absoluto medio 0,14, Claude 0,11 y PersonalityMap 0,07, frente a medias por participante de 0,29 en legos y 0,20 en expertos. GPT-4o quedó en el percentil 95,67 frente a legos y 70,22 frente a expertos; Claude, en 100 y 95,22; PersonalityMap, en 100 frente a ambos. Los win rates, calculados sobre los pares realmente contestados por cada persona, fueron 90,94%/69,85% para GPT-4o, 97,64%/86,40% para Claude y 100%/99,26% para PersonalityMap frente a legos/expertos. Esto respalda que una consulta de estos sistemas suele ser más precisa que un individuo en esta tarea. No prueba una comprensión general de la personalidad: predice parámetros de ítems conocidos y la comparación individual da a la IA 249 pares, pero a cada humano solo 30.

Cuando se agrega por ítem mediante la mediana, la ventaja cambia. El error medio fue 0,164 para legos, 0,086 para expertos, 0,140 para GPT-4o, 0,104 para Claude y 0,072 para PersonalityMap. Las correlaciones entre predicción y valor empírico fueron 0,88, 0,90, 0,78, 0,80 y 0,91, respectivamente. El test de buckets (<−0,1, −0,1 a 0,1, >0,1) no encontró diferencias, χ²(4)=6,42, p=.170. Sin embargo, las pruebas publicadas de error tratan como independientes cinco medidas sobre los mismos 249 pares: usan Kruskal–Wallis y llaman “Dunn” a comparaciones Mann–Whitney con Paired=False. Una comprobación editorial emparejada produce Friedman χ²(4)=96,59, p<.001 y Wilcoxon-Holm: confirma la superioridad general de PersonalityMap y Claude sobre GPT-4o, pero cambia dos conclusiones límite, GPT-4o frente a legos pasa a p=.047 y PersonalityMap frente a expertos a p=.047, mientras expertos frente a Claude queda en p=.053. Para los buckets, el Cochran Q emparejado sigue sin ser significativo, Q(4)=8,70, p=.069. Estas comprobaciones no sustituyen una revisión estadística formal, pero muestran que parte del detalle inferencial depende de haber ignorado el emparejamiento.

Los análisis añadidos tras la primera revisión por pares prueban 30 consultas no sembradas de GPT-4o a temperatura 1 y SurveyBot3000. La mediana de las 30 consultas mejora la correlación agregada de GPT-4o hasta 0,88, sin mejorar de forma uniforme las otras métricas. SurveyBot3000 parece competitivo, pero fue entrenado con SAPA y por tanto con el propio conjunto usado como test; los autores reconocen que no permite distinguir generalización de memorización. La lectura del peer review transparente confirma que el título se estrechó de “entender personalidad” a “predecir correlaciones entre ítems” y que los revisores pidieron reducir las aplicaciones especulativas.

El artefacto OSF permite reproducir las cifras principales, pero no es una reproducción de extremo a extremo. Incluye dos Excel procesados, dos scripts monolíticos y dos JSON de verdad, sin README, requirements, lockfile, CLI ni código para recopilar respuestas de los LLM o ejecutar PersonalityMap. Los paths son XXX/...; tras reconstruirlos y usar las versiones declaradas en el reporting summary, el script principal termina y reproduce las tablas. El script adicional también termina, pero fija num_bootstraps = 10 aunque el artículo declara 10.000, genera intervalos distintos de los publicados y produce CIs nan para dos condiciones improcedentes. Ambos scripts imprimen el p-valor como si fuera también el estadístico W de Shapiro o F de Levene. PersonalityMap sigue siendo propietario: dos autores lo crearon y uno fundó además Positly y GuidedTrack, un conflicto declarado; la afirmación de que ningún ítem de test entró en su entrenamiento no puede comprobarse con el depósito.

La contribución defendible es demostrar que los pronósticos sintéticos de correlaciones pueden ser baratos y sorprendentemente precisos, y que agregar expertos humanos sigue siendo competitivo con LLM generales. El alcance correcto es la predicción de correlaciones lineales, transversales y de autoinforme en un inventario inglés y una distribución de test artificialmente estratificada. No valida selección de personal, diagnóstico, marketing, evaluación psicológica automatizada ni sustitución de datos humanos. Tampoco convierte una correlación predicha en evidencia causal: los propios autores señalan que cualquier uso científico debe confirmarse finalmente con personas reales y que faltan estimaciones de incertidumbre para PersonalityMap.

Research question

How accurately do laypeople, academic experts, GPT-4o, Claude 3 Opus, and a specialized network predict the empirical correlations between pairs of SAPA items, both as individual predictors and after aggregating their responses by median?

Method

Preregistered comparative study on 249 pairs of 103 SAPA items, stratified into three correlation bands. Each of 254 laypeople and 272 experts rated 30 randomly assigned pairs on a scale from −1 to 1. GPT-4o and Claude 3 Opus produced three predictions per pair at temperature 0 with a 671-token prompt; PersonalityMap produced one. RQ1 compares one run of each LLM selected with seed 1 against each individual using bootstrap percentiles and win rates. RQ2 aggregates by median within each condition and compares absolute error, prediction–truth correlation, and accuracy across three buckets. Exploratory analyses added SurveyBot3000 and 30 runs of GPT-4o at temperature 1. The editorial audit read and rendered the article and all supplements, reviewed the peer review and reporting summary, inspected both Excel files with 249 prediction columns, ran the two scripts with the declared versions, and contrasted the independent tests with paired alternatives.

Sample: 254 laypeople recruited by Positly (mean age 46.35; SD 11.83; 56% men; $1.80, approximately $8.40/h) and 272 academic experts recruited through lists and networks (mean age 33.86; SD 8.12; 52% men; 36% professors, 20% PhDs who are not professors, and 44% graduate students; $5 gift card or donation). Each person received 30 pairs; laypeople answered a median of 30, minimum 4 per person, and each pair received at least 16 responses. Experts completed 30 per person and each pair received at least 18. Each LLM at temperature 0 produced 3×249 predictions; PersonalityMap and SurveyBot3000, 1×249; GPT-4o at temperature 1, 30×249.

Findings

  • The mean absolute error per individual predictor was 0.29 for laypeople, 0.20 for experts, 0.14 for GPT-4o, 0.11 for Claude 3 Opus, and 0.07 for PersonalityMap.
  • The run selected with seed 1 was GPT2 for GPT-4o and Claude1 for Claude; the code reproduces the published percentiles.
  • GPT-4o reached percentiles 95.67 against laypeople and 70.22 against experts; Claude, 100 and 95.22; PersonalityMap, 100 against both.
  • Win rates against laypeople/experts were 90.94%/69.85% for GPT-4o, 97.64%/86.40% for Claude, and 100%/99.26% for PersonalityMap, all above the published 50%.
  • After aggregating by median, the mean errors were 0.164 laypeople, 0.086 experts, 0.140 GPT-4o, 0.104 Claude, and 0.072 PersonalityMap.
  • The article reports Kruskal–Wallis H(4)=90.84, p<.001 and independent post-hoc tests; the code reproduces them exactly.
  • The paired audit obtains Friedman χ²(4)=96.59, p<.001; the overall hierarchy persists, but GPT-4o–layperson and PersonalityMap–expert change to p=.047, and expert–Claude remains p=.053 after Holm.
  • The aggregated correlations with the truth were 0.879 laypeople, 0.895 experts, 0.783 GPT-4o, 0.803 Claude, and 0.908 PersonalityMap.
  • A paired bootstrap of differences confirms that laypeople and experts correlate more with the truth than both LLMs; PersonalityMap does not differ from laypeople or experts, and outperforms both LLMs on this metric.
  • Accuracies by bucket were 196 laypeople, 193 experts, 183 GPT-4o, 184 Claude, and 203 PersonalityMap out of 249; the published χ² gives p=.170 and paired Cochran Q p=.069.
  • Thirty runs of GPT-4o at temperature 1 raise the median correlation to 0.88 and its aggregated mean error to 0.117, but do not uniformly improve percentile, win rate, or buckets.
  • Individual reliability ICC(2,1) was 0.874 for GPT-4o at temperature 0, 0.962 for Claude, and 0.826 for GPT-4o at temperature 1; when aggregating k runs, ICC(2,k) was 0.954, 0.987, and 0.993.
  • SurveyBot3000 reaches aggregated error 0.079 and correlation 0.877, but its training included SAPA and directly contaminates this test.
  • The main Excel contains exactly 254 layperson rows, 272 expert rows, three GPT, three Claude, and one PersonalityMap; the additional one adds one SurveyBot3000 and thirty high-temperature GPT.
  • After reconstructing paths XXX and installing the versions from the reporting summary, the main script terminates and reproduces the published tables, figures, and counts.
  • The additional script sets only 10 bootstraps for the correlation intervals; run without changes it produces CIs different from Supplementary Table 6 and does not reproduce that inferential analysis.
  • The code calls Dunn to pingouin.pairwise_tests with between='Condition', which returns independent Mann–Whitney U tests and declares Paired=False.
  • The final title and limitations were narrowed during peer review to reflect that the task is correlation between items, not general understanding of personality.

Limitations

  • The test covers only linear correlations between English SAPA self-report items and a cross-sectional measurement.
  • The 249 pairs were stratified by magnitude and sign; the distribution does not represent the natural prevalence of correlations between items.
  • The values called truth are empirical estimates from SAPA, not error-free parameters; the effective size and uncertainty may vary by pair.
  • Each human predicted 30 pairs, while the systems predicted 249; the individual average error does not have the same precision or coverage.
  • The AI received an extensive expert prompt with chain-of-thought, tree-of-thought, counterarguments, emotional stakes, and a fictitious $20 payment claim; humans received a different interface and much lower real payments.
  • No accuracy incentive was provided to laypeople or experts, a limitation acknowledged by the authors.
  • GPT-4o and Claude may have seen SAPA during pretraining; no training data are available to rule this out.
  • SurveyBot3000 did see SAPA, so its result is not a valid test of generalization.
  • PersonalityMap is proprietary; the repository does not allow inspection of weights, training data, or verification of the exclusion of the 103 evaluated items.
  • Two authors created PersonalityMap and Spencer Greenberg founded Positly and GuidedTrack; the conflict is declared, but increases the importance of independent validation.
  • Human medians combine at least 16/18 and typically 30 or more judgments per pair; LLMs at temperature 0 only three and PersonalityMap one, so the aggregation budget is not equivalent.
  • Kruskal–Wallis, Mann–Whitney, and χ² of independence ignore that each condition is evaluated on the same 249 pairs; paired or repeated-measures methods are required.
  • Inferring differences between correlations by overlapping separate CIs is not a direct test of dependent correlations that share the same truth and the same items.
  • The percentile bootstrap keeps the human distributions fixed and resamples AI questions; it does not propagate all the uncertainty from both sides of the comparison.
  • The code prints the p-values as if they were also the Shapiro W and Levene F statistics; the inferential reporting of assumptions is erroneous.
  • The additional script uses 10 bootstraps despite declaring 10,000 and does not reproduce the published intervals; it also does not set a specific seed for those intervals.
  • There is no README, requirements, lockfile, tests, CI, entrypoint, or manifest connecting outputs to tables and figures.
  • The scripts contain paths XXX, write multiple derivatives and figures by side effects, and require manual reconstruction of the environment.
  • The repository contains processed predictions, not raw LLM responses, API logs, costs, timestamps, or generation code; parsing or context contamination cannot be audited.
  • No uncertainty intervals per PersonalityMap prediction or out-of-distribution calibration are published.
  • The human sample is a convenience sample and the benchmark comes from a WEIRD context; other languages, cultures, item formats, and data sources may alter the comparison.
  • Accuracy in self-report correlations may reflect shared method bias, not only underlying psychological relationships.
  • Applications to hiring, health, marketing, or diagnosis are mentioned without evaluating decisions, individuals, clinical outcomes, fairness, safety, or incremental validity.
  • Predicted correlations do not allow inferring causality and the specialized system itself only models linear relationships.

What the study does not establish

  • It does not demonstrate that GPT-4o, Claude, or PersonalityMap understand personality in a general psychological sense.
  • It does not validate these systems as substitutes for psychometric tests administered to people.
  • It does not authorize hiring, diagnostic, treatment, segmentation, or individual evaluation decisions.
  • It does not demonstrate that PersonalityMap generalizes to items, populations, cultures, or instruments outside SAPA.
  • It does not separate psychological knowledge, prior exposure to SAPA, statistical calibration, and learning of self-report biases as sources of performance.
  • It does not establish robust differences between all conditions with the published inferential analysis, because it ignores pairing by item.
  • It does not offer a complete reproduction of the collection of predictions or of the additional intervals with the released artifact.
  • It does not convert estimated associations into causal mechanisms or eliminate the need for confirmation with real human data.

Traceability

Scope: Full text

Version: Communications Psychology 3, article 23; published 12 February 2025; main article 12 pages plus Supplementary Information, transparent peer review and reporting summary

Consulted source: https://doi.org/10.1038/s44271-025-00205-w

Review: Codex full-text, peer-review, spreadsheet, code and statistical robustness audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o snapshot gpt-4o-2024-05-13, temperature 0, three runs
  • Claude 3 Opus snapshot claude-3-opus-20240229, temperature 0, three runs
  • PersonalityMap proprietary deterministic deep neural network
  • GPT-4o snapshot gpt-4o-2024-05-13, temperature 1, thirty unseeded runs (additional analysis)
  • SurveyBot3000 fine-tuned all-mpnet-base-v2 model (additional contaminated comparison)

Instruments and metrics

  • 249 pairs from 103 SAPA Personality Inventory self-report items
  • Human correlation-estimation slider from −1 to 1 in 0.02 increments
  • Correlation primer and comprehension questions
  • Mean absolute prediction error
  • Percentile rank and item-matched win rate
  • Median aggregation by item
  • Pearson prediction–truth correlation with Fisher-z bootstrap intervals
  • Three-bin directional/magnitude classification
  • ICC(2,1) and ICC(2,k) for repeated LLM predictions

Data used

  • SAPA empirical item correlations from Condon et al. (2017)
  • OSF main processed dataset: 533 observations × 271 columns, including 249 prediction columns
  • OSF additional processed dataset: 564 observations × 271 columns
  • OSF Truth.json and EmpiricalCorUnweighted.json
  • OSF analysis scripts Code.py for main and additional conditions
  • Transparent peer-review file and Nature Portfolio reporting summary

Evidence and location

  • Design, sample, SAPA pairs, models, prompt, and preregistration: Main article, pp. 2–4, research questions and Methods; Supplementary Information, pp. 1–9
  • Errors, percentiles, win rates, and aggregated comparisons: Main article, pp. 4–8, Tables 1–5 and Figures 1–5
  • GPT-4o at temperature 1, SurveyBot3000, and ICC: Main article, pp. 7–8; Supplementary Information, pp. 9–14, Tables 2–6
  • Limitations, applications, conflicts, and availability: Main article, pp. 8–12, Discussion, Limitations, Practical implications, Competing interests and Data/Code availability
  • Changes requested by reviewers and narrowing of the title and claims: Transparent peer review, Version 0 reviewer reports and Version 1 decision letter/rebuttal
  • Structure of the Excel files and reproduction of the main script: OSF osf.io/kzgy7, Data and Code/Data.xlsx plus Code.py, audited and executed 15 Jul 2026
  • Bootstrap of 10, CIs not reproduced, and mislabeled independent tests: OSF additional Code.py lines 578–657 and 764–792; Full Data Processed.xlsx; executed 15 Jul 2026
  • Robustness with paired tests: Editorial reanalysis of the released 249 paired item predictions, Friedman/Wilcoxon-Holm, Cochran Q/McNemar and paired bootstrap, 15 Jul 2026