This 100-page preprint proposes evaluating LLM behavioral tendencies through situational judgment tests (SJTs) rather than relying only on HEXACO self-reports. GPT-4.1 generates detailed synthetic personas from demographics, memoir seeds, and eight police archetypes. Experts define 20 base scenarios, and GPT-4.1 expands controlled combinations of age, gender, race, ambiguity, threat, urgency, authority, time, and ethical tension into 4,000 SJTs. Each situation offers six responses authored to represent one HEXACO dimension; an LLM judge detects and rewrites options with trait bleed. The main public run crosses 500 personas, 300 SJTs, and five repetitions using Gemma-3-4B, producing 750,000 responses. Analyses compare SJTs, HEXACO self-reports, TruthfulQA, and EmoBench through distributions, correlations, Jensen-Shannon divergence, ICC, regressions, and a Bayesian MIRT model.
SJT profiles show stronger relationships with external benchmarks than several self-report profiles. For example, SJT Agreeableness correlates .70 with EmoBench versus .51 for HEXACO Agreeableness. Personas yield repeatable shifts and different archetype effects; benchmark distributions have near-zero JS divergence and ICC of .95/.90. In a pilot MIRT fit with 250 personas and 150 items, 76% of scenarios have discrimination above 1.0, and five of six latent traits predict their corresponding option; Emotionality does not. The most important scope result is the variance decomposition: persona identity explains 3.75%, scenario 38.93%, and residual variation 57.32%. A persona-label permutation test gives p=.002, so the persona effect is detectable but small relative to context and unexplained variation.
The defensible contribution is methodological: decisions in context expose conditional variation that transparent questionnaires can miss. The study does not establish human personality, subjective states, or dominant stable dispositions. Each option is authored to express a HEXACO label, so recovered structure partly measures compliance with that synthetic mapping. GPT-4.1 participates in persona generation, SJT generation, trait-bleed correction, and evaluation, creating partial circularity. Only 30 SJTs receive ratings from a psychologist and a patrol officer, and the appendix states that independent expert raters were not employed; several kappa values are zero or near zero. TruthfulQA and EmoBench are model benchmarks rather than human behavior. Adjusted R-squared values of .863-.993 are in-sample associations among scores derived from the same responses, not held-out prediction. Nor can 750,000 crossed response rows be treated as independent observations.
The release is substantial: four public Hugging Face repositories are open, Apache-2.0 licensed, and their counts match the documentation; the MIT repository contains generation, response, and analysis code. It still lacks scientific CI, a real test suite, and a manifest mapping every table to an exact commit, command, environment, and hash. More seriously, a versioned shell script exposes a Hugging Face access token. The token was neither used nor reproduced in this audit; it should be treated as compromised, revoked, and purged from history. Demographic attributes in police scenarios are synthetic, but they can still encode stereotypes, and a bias rubric is not deployment validation for high-impact settings.