The paper proposes the Persona Brainstorm Audit (PBA), a procedure for detecting potentially stereotypical associations in LLM-generated persona lists. This review uses arXiv v2, revised on 24 February 2026: a 27-page ACM manuscript that still contains the template text “Make sure to enter the correct conference title” and a placeholder DOI. No definitive publication or official repository was found; the paper itself postpones release of profiles, mappings, and code until publication.
The base prompt asks for “20 diverse user profiles” in JSON per call, with eight fields: name, gender, ethnicity, sexual orientation, social class, education, occupation, and top personal interest. The study nominally generates 10,000 profiles from each of 12 models: nine OpenAI aliases, GPT-3.5, GPT-4, GPT-4o, three GPT-4.1 variants, and three GPT-5 variants, and three Mistral releases. Temperature is 1. Values are lowercased, stemmed, deduplicated, and consolidated into categories by GPT-5, reportedly followed by human validation. The reduction is extreme: for example, GPT-5's 1,128 ethnicity strings become six categories; 689 education strings become five; 1,384 occupations become 18; and 1,501 interests become 15. Only the 50 most frequent names across models are retained for name-based analysis.
PBA crosses four identity axes, name, gender, ethnicity, and sexual orientation, with four social dimensions, class, education, occupation, and interest, builds 16 contingency tables, and computes Cramér's V. It then transforms V through degree-of-freedom-dependent thresholds into severity scores: 0–0.33 small, 0.33–0.66 medium, 0.66–1 high, and >1 very high. The exact algorithm for this second normalization is not specified mathematically. Cramér's V measures association, not harm, discrimination, or unfairness; an association may be semantically expected or vary with margins and analyst-created categories. The paper also calls the normalized metric “bounded” although its results exceed 1.
In the main analysis, GPT-4o has the lowest mean score (0.678), followed by Ministral-3B (0.699), GPT-4 and GPT-5 nano (both 0.707). GPT-4.1 mini (0.969), GPT-5 (0.950), GPT-5 mini (0.920), and Mistral-medium (0.893) rank highest. Name associations are largest on average (1.113), while occupation and interest concentrate high scores. Heatmaps show non-heterosexual identities clustering in Creative & Design and appearing less in Engineering, gender-linked occupations, and differences when gender × sexual orientation × social class are combined. The nurse drill-down assigns more than 90% of nurses to women in almost every model; GPT-4.1 nano reaches 100%. These distributions document generated patterns worth inspecting, but the aggregate score alone does not determine which patterns are harmful.
The paper presents GPT-3.5 → GPT-4 → GPT-4o → GPT-4.1 → GPT-5 as a longitudinal trajectory and concludes that bias declines and resurfaces. Without exact snapshots, inference dates, or comparable checkpoints, this is a cross-sectional comparison of distinct products, not controlled longitudinal tracking of one system. Paired t-tests treat the 16 correlated dimensions as replicates when comparing models, with unclear independence. The analyzed sample is also not exactly 120,000 after deduplication: 3,778 profiles are removed, leaving 116,222; Mistral-medium loses 2,060 (20.6%) compared with only 3 for GPT-5. Unequal deduplication changes the distributions and removes a direct signal of model collapse or concentration.
The sample-size analysis compares 5k, 10k, 15k, and 20k profiles on six models and reports high ICCs and correlations for most dimensions, but it does not state whether samples are independent or nested, and provides no repeated samples or intervals. Under the UX-researcher role, most severity labels persist. Under the debiasing prompt, stability drops sharply: recomputing the 16 rows in Table 5 gives means ICC(C,1)=0.381, ICC(A,1)=0.361, Spearman=0.422, Kendall=0.306, and severity difference=0.312. The printed “Mean” row, 0.320, 0.319, −0.200, −0.200, 0.167, is not the mean of the rows. Nevertheless, the abstract and conclusion claim stability under debiasing prompts. The base prompt already requires “diverse” profiles, so the intervention is not compared with a neutral condition; and a lack of p<0.05 with few models does not demonstrate no effect or equivalence.
Human validation is exploratory: nine evaluators compare only two Gender × Occupation charts selected because GPT-5 mini scores 1.134 and GPT-5 nano 0.779. Everyone judges the former more biased. This provides face validity for one obvious contrast, but does not validate all 16 dimensions, the thresholds, degree-of-freedom comparability, or regulatory use. The paper does not report ethics review, consent, exact DEI/non-DEI composition, compensation, order randomization, or inter-rater agreement.
The defensible contribution is a simple protocol for generating open categorical data and locating associations for human auditors to investigate, with useful macro-to-micro drill-down examples. It does not establish that the normalized scale validly and comparably measures fairness, that it is robust to debiasing prompts, or that it supports model rankings, deployment thresholds, or certification. PBA should be presented as an exploratory signal-detection instrument rather than a validated fairness metric.