The reference version is the paper published in Findings of IJCNLP-AACL 2025. It administers the 60 BFI-2 items one prompt at a time, requesting a numerical 1–5 answer, to systems labeled Llama 3 8B, Mistral 7B, MythoMax L2 13B, Gemma 9B, Qwen 7B, and StripedHyena 7B. The paper reports 21 temperatures from 0 to 2 and one answer per item, model, and temperature: 7,560 cells with no replications. It incorrectly says increments of 1; code and CSVs confirm 0.1 steps. It reports Kruskal–Wallis differences for Extraversion, Agreeableness, Conscientiousness, and Openness but not Neuroticism, plus a negative temperature correlation with Neuroticism and a positive one with Extraversion. Its cluster places Gemma–Mistral together, Llama–StripedHyena in another branch, and Qwen at the opposite edge. The OSF audit, however, prevents a confirmatory interpretation. The supposed Qwen 7B is actually `Qwen/Qwen1.5-72B-Chat` in both code and all 1,260 CSV rows, so the sample is not six comparable 7B–13B models. The executable BFI-2 reverse-key list omits items 3, 4, 8, 9, 37, 42, 47, 48, and 58 and wrongly reverses item 46. Another script header prints the correct key, but calculations import the wrong one. Four domains and every associated mean, test, regression, figure, and cluster are affected; only Openness has the complete correct key. Released code and six CSVs also fail to reproduce the tables. The released key yields H=47.0473/85.1752/79.0852/13.7920/72.8568 rather than 40.7803/65.3067/63.0415/9.2691/58.1957. Released temperature correlations are r=-.4821 for Neuroticism and .5194 for Extraversion rather than -.5904 and .5021. Correct scoring makes Neuroticism H=44.8853, p=1.53e-8, contradicting the paper's central null. The aggregate Extraversion association falls to r=.4438; excluding the mislabeled 72B Qwen makes it r=.3757, p=.0933 and non-significant. There are also 265 NaN responses among 7,560 cells: 200 for StripedHyena and 51 for Qwen. StripedHyena loses as many as 36 of 60 items at one temperature. Pandas silently averages nonmissing items, producing unequal denominators without a documented missingness rule. The regression is not multiple linear regression by model: code first averages all six systems at each temperature and fits one line to only 21 ecological means. Kruskal–Wallis treats the 21 ordered conditions from the same questionnaire as independent samples for each model. One generation per cell cannot estimate sampling noise, test–retest reliability, or stability; changing temperature is not replication. The paper omits top-k=60, top-p=.8, repetition penalty=1.1, max tokens=20, and conversion of errors or invalid formats to NaN. It provides no factor structure, reliability, invariance, post-hoc comparison, or confidence intervals. A dendrogram of six profiles confounded by size, data, tuning, tokenizer, vocabulary, and attention cannot identify architectural predisposition. The final evidence is an open exploratory numeric-response dataset whose artifact exposes serious errors; it does not establish emergent or stable traits, temperature causality, architecture effects, or a governance basis.
Research question
Do six systems labeled as 7B-13B LLMs differ in BFI-2 responses, does their aggregated mean change with temperature, and do they form descriptively clusterable profiles?