Measure what Matters: Psychometric Evaluation of AI with Situational Judgment Tests

Evaluation and psychometric validity2025arXivApproved editorial review

Authors: Alexandra Yost, Shreyans Jain, Shivam Raval, Grant Corser, Allen Roush, Nina Xu, Jacqueline Hammack, Ravid Shwartz-Ziv, Amirali Abdullah

Keywords: Large Language Models, Situational Judgment Tests, Psychometrics, Synthetic personas, HEXACO, Multidimensional item response theory, Behavioral evaluation, Persona conditioning

Source: Open primary source (opens in a new tab)

9
Authors
9
Findings
15
Limitations
6
Evidence

Editorial summary

English

This 100-page preprint proposes evaluating LLM behavioral tendencies through situational judgment tests (SJTs) rather than relying only on HEXACO self-reports. GPT-4.1 generates detailed synthetic personas from demographics, memoir seeds, and eight police archetypes. Experts define 20 base scenarios, and GPT-4.1 expands controlled combinations of age, gender, race, ambiguity, threat, urgency, authority, time, and ethical tension into 4,000 SJTs. Each situation offers six responses authored to represent one HEXACO dimension; an LLM judge detects and rewrites options with trait bleed. The main public run crosses 500 personas, 300 SJTs, and five repetitions using Gemma-3-4B, producing 750,000 responses. Analyses compare SJTs, HEXACO self-reports, TruthfulQA, and EmoBench through distributions, correlations, Jensen-Shannon divergence, ICC, regressions, and a Bayesian MIRT model.

SJT profiles show stronger relationships with external benchmarks than several self-report profiles. For example, SJT Agreeableness correlates .70 with EmoBench versus .51 for HEXACO Agreeableness. Personas yield repeatable shifts and different archetype effects; benchmark distributions have near-zero JS divergence and ICC of .95/.90. In a pilot MIRT fit with 250 personas and 150 items, 76% of scenarios have discrimination above 1.0, and five of six latent traits predict their corresponding option; Emotionality does not. The most important scope result is the variance decomposition: persona identity explains 3.75%, scenario 38.93%, and residual variation 57.32%. A persona-label permutation test gives p=.002, so the persona effect is detectable but small relative to context and unexplained variation.

The defensible contribution is methodological: decisions in context expose conditional variation that transparent questionnaires can miss. The study does not establish human personality, subjective states, or dominant stable dispositions. Each option is authored to express a HEXACO label, so recovered structure partly measures compliance with that synthetic mapping. GPT-4.1 participates in persona generation, SJT generation, trait-bleed correction, and evaluation, creating partial circularity. Only 30 SJTs receive ratings from a psychologist and a patrol officer, and the appendix states that independent expert raters were not employed; several kappa values are zero or near zero. TruthfulQA and EmoBench are model benchmarks rather than human behavior. Adjusted R-squared values of .863-.993 are in-sample associations among scores derived from the same responses, not held-out prediction. Nor can 750,000 crossed response rows be treated as independent observations.

The release is substantial: four public Hugging Face repositories are open, Apache-2.0 licensed, and their counts match the documentation; the MIT repository contains generation, response, and analysis code. It still lacks scientific CI, a real test suite, and a manifest mapping every table to an exact commit, command, environment, and hash. More seriously, a versioned shell script exposes a Hugging Face access token. The token was neither used nor reproduced in this audit; it should be treated as compromised, revoked, and purged from history. Demographic attributes in police scenarios are synthetic, but they can still encode stereotypes, and a bias rubric is not deployment validation for high-impact settings.

Español

Este preprint de 100 páginas propone evaluar tendencias conductuales de LLM mediante situational judgment tests (SJT) en vez de depender solo de autoinformes HEXACO. GPT-4.1 genera personas sintéticas detalladas a partir de demografía, memorias y ocho arquetipos de policía. Expertos definen 20 escenarios base y GPT-4.1 expande combinaciones controladas de edad, género, raza, ambigüedad, amenaza, urgencia, autoridad, hora y tensión ética hasta 4.000 SJT. Cada situación ofrece seis respuestas escritas para representar una dimensión HEXACO; un juez LLM detecta y reescribe opciones con trait bleed. La ejecución principal pública cruza 500 personas, 300 SJT y cinco repeticiones con Gemma-3-4B: 750.000 respuestas. El análisis compara SJT, autoinformes HEXACO, TruthfulQA y EmoBench mediante distribuciones, correlaciones, Jensen-Shannon, ICC, regresiones y un modelo MIRT bayesiano.

Los SJT muestran patrones más relacionados con benchmarks externos que varios autoinformes. Por ejemplo, Agreeableness-SJT correlaciona .70 con EmoBench frente a .51 para Agreeableness-HEXACO. Las personas producen desplazamientos estables entre repeticiones y efectos distintos por arquetipo; los benchmarks presentan JS casi cero e ICC .95/.90. En un MIRT piloto con 250 personas y 150 ítems, 76% de los escenarios tiene discriminación superior a 1,0 y cinco de seis rasgos latentes predicen su opción correspondiente; Emotionality no lo hace. El resultado más importante para interpretar alcance es la descomposición: identidad de persona explica 3,75% de la varianza, escenario 38,93% y residuo 57,32%. La permutación de etiquetas obtiene p=.002, por lo que el efecto de persona es detectable, pero pequeño frente al contexto y la variación no explicada.

La contribución defendible es metodológica: decidir dentro de escenarios revela variación condicional que un cuestionario transparente puede ocultar. No demuestra personalidad humana, subjetividad o disposiciones dominantes. Cada opción está escrita para expresar una etiqueta HEXACO, de modo que recuperar estructura también mide obediencia a ese mapeo sintético. GPT-4.1 participa en personas, SJT, corrección de trait bleed y evaluación, creando circularidad parcial. Solo 30 SJT reciben anotación de un psicólogo y un agente, y el apéndice reconoce que no hubo validadores expertos independientes; varios kappa son cero o casi cero. TruthfulQA y EmoBench son benchmarks de modelos, no conducta humana. Los R² ajustados .863-.993 son asociaciones in-sample entre scores derivados de las mismas respuestas, no predicción holdout. Tampoco se pueden tratar 750.000 filas cruzadas como observaciones independientes.

La release es sustancial: los cuatro repositorios públicos de Hugging Face son abiertos, Apache-2.0 y sus conteos coinciden con la documentación; el repositorio MIT contiene código de generación, respuesta y análisis. Aun así, faltan CI científico, una suite de tests y un manifiesto que conecte cada tabla con un commit, comando, entorno y hash. Además, un script versionado expone un token de acceso de Hugging Face. El token no se usó ni se reproduce aquí: debe considerarse comprometido, revocarse y eliminarse también del historial. Los atributos demográficos en escenarios policiales son sintéticos, pero pueden codificar estereotipos; una rúbrica de sesgo no equivale a validación para despliegue de alto impacto.

Research question

Do situated SJTs measure stable behavioral tendencies conditioned by personas with greater external coherence and psychometric validity than self-reports from human inventories applied directly to LLMs?

Method

Generation of personas and 4,000 synthetic SJTs, curation of trait bleed, Gemma-3-4B responses to 500 personas across 300 SJTs with five repetitions, comparison with HEXACO/TruthfulQA/EmoBench, human review and review by two LLM judges, JS/ICC stability, correlations, regressions, and Bayesian multidimensional MIRT.

Sample: The central execution contains 500 personas x 300 scenarios x 5 responses = 750,000 dependent rows. The MIRT is fit on a pilot of 250 personas and 150 items. The human evaluation covers 30 SJTs with two domain experts and 55 synthetic personas; there is no real human behavior cohort as ground truth.

Findings

  • The 100 pages, including prompts, tables 1-59 and appendices, were rendered and visually inspected.
  • SJT-Agreeableness correlates .70 with EmoBench compared to .51 for HEXACO-Agreeableness.
  • The benchmark distributions are stable across repetitions, with ICC .95 and .90.
  • Persona identity explains 3.75% of the variance, the scenario 38.93%, and the residual 57.32%.
  • The permutation of persona labels reports p=.002.
  • In the MIRT, 76% of 150 items have discrimination greater than 1.0.
  • Five of six latent traits predict their option; Emotionality does not reach significance.
  • The four dataset repositories are public and their sizes match README/DATASETS.md.
  • A Hugging Face token is exposed in a versioned shell script of the official repository.

Limitations

  • The SJT options incorporate by construction the HEXACO label that is then attempted to be recovered.
  • GPT-4.1 participates in generation, correction, and evaluation, creating circularity.
  • Only 30 SJTs receive human annotation and several kappa are zero or near zero.
  • There is no independent panel of experts validating the entire instrument.
  • The persona effect is small compared to the scenario and the residual.
  • The external benchmarks are model tests, not human behavioral results.
  • The very high R² are calculated in-sample without holdout personas or scenarios.
  • The 750,000 responses are crossed and nested, not independent observations.
  • The MIRT uses a pilot subset and an implementation with one option fixed to zero for identification.
  • Most results come from Gemma-3-4B and an English police domain.
  • Long conversations, deployment, cultures, languages, or real human decisions are not validated.
  • Attributes of race, gender, and age may introduce stereotypes even in synthetic data.
  • The release lacks scientific CI and exact table-to-command traceability.
  • Proprietary models and service configurations are not immutable snapshots.
  • The public repository exposes a credential that must be revoked and purged.

What the study does not establish

  • It does not establish human personality, consciousness, emotions, or subjectivity in the models.
  • It does not demonstrate that the persona is the dominant cause of the responses.
  • It does not validate HEXACO as a natural internal ontology of an LLM.
  • It does not convert correlations with EmoBench or TruthfulQA into human ecological validity.
  • It does not demonstrate out-of-sample prediction with the reported R².
  • It does not demonstrate safety or impartiality for police or selection uses.
  • It does not automatically generalize to other models, languages, cultures, or domains.
  • It does not eliminate the need for independent human validation.

Traceability

Scope: Full text

Version: arXiv:2510.22170v2, submitted 25 October 2025, revised 9 May 2026, 100 pages

Consulted source: https://arxiv.org/abs/2510.22170

Review: Codex complete 100-page full-text, all-page visual, TeX source, SJT/HEXACO construct, MIRT, variance, annotator, GitHub code, Hugging Face dataset, security and reproducibility audit; summaries written from the complete paper rather than abstract extraction, 2026-07-18

Approval: Codex fidelity pass, 2026-07-18

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4.1 for persona and SJT generation, correction and part of evaluation
  • GPT-4.1-mini for selected evaluation stages
  • Claude-3.5-Sonnet for cross-generator and judge comparisons
  • Gemma-3-4B-Instruct for the main 750,000-response experiment
  • Qwen-2.5-7B-Instruct
  • Qwen3-0.6B embedding model
  • Llama-3.1-8B-Instruct

Instruments and metrics

  • HEXACO-100 self-report inventory
  • Synthetic six-option Situational Judgment Tests mapped to HEXACO
  • Multidimensional item response theory
  • TruthfulQA
  • EmoBench
  • Jensen-Shannon divergence
  • Intraclass correlation coefficient
  • Two complementary SJT quality rubrics
  • Cohen kappa and mean absolute deviation
  • Lexical and embedding diversity measures

Data used

  • thoughtworks/psychometric_personas: 8,500 base and 500 analysis personas
  • thoughtworks/psychometric_SJTs: 4,000 raw synthetic SJTs
  • thoughtworks/psychometric_sjts_analysis: 1,000 expanded and 300 analysis SJTs
  • thoughtworks/gemma_psychometrics_personas_responses: 750,000 main analysis_sjt rows
  • Thirty human-annotated SJT items
  • Fifty-five annotated synthetic personas
  • British parliamentarian persona transfer set

Evidence and location

  • Full text, tables, figures, prompts, cases, and appendices: arXiv:2510.22170v2, all 100/100 pages rendered and visually inspected
  • Design, main results, and limits: Main paper pages 1-13 and appendices A-Z pages 16-100
  • MIRT, variance, stability, and construct validity: Appendices M-Q, PDF pages 50-65
  • Code, license, secret, and reproducibility: Official GitHub repository amir-abdullah-thoughtworks/psychometrics_for_LLMs at commit 170685fa33fb49e8fb065bf38ed96f15bb53ae4b
  • Existence, license, and dataset counts: Hugging Face Dataset Viewer and Hub metadata for four thoughtworks repositories checked 18 July 2026
  • Construct, statistical, data, security, and claims audit: reports/verification/article-233-sjt-persona-construct-mirt-dataset-secret-and-reproducibility-audit.json