This five-page WWW Companion 2025 paper administers five reworded personality inventories to GPT-4, GPT-4o-mini, Llama-3-8B-Instruct, Llama-3.1-8B-Instruct, and Llama-3.2-3B-Instruct. GPT-4o rewrites the items; all-MiniLM-L6-v2 requires cosine similarity of at least 0.7 to the original, and the authors state that items below the threshold are reconstructed under human supervision. Items are randomized, sent in batches of ten, and repeated 100 times at temperature 0 for OpenAI and 0.01 for Llama. Table 2 releases only the mean by model, inventory, and dimension. No code, complete prompts, reworded forms, raw responses, or reproducible configuration was located. The protocol neither removes nor measures training-data contamination. A semantically close paraphrase may remain recognizable and may itself have been generated by a model familiar with the instrument; a similarity threshold measures textual resemblance, not presence in training data. It also does not validate preservation of scoring, factor structure, or psychometric meaning. The system prompt frames the model as a helpful assistant and asks it to rate itself, so responses may reflect alignment, format obedience, and stereotypes about a helpful assistant rather than latent traits. Instrument selection contains conceptual errors. HEXACO-100 measures six factors; the paper drops Honesty-Humility and forces Emotionality and the remaining domains into five Big Five labels. The 60-item measure called NEO-PI-R is actually IPIP-NEO-60, a public-domain representation of NEO PI-R. TIPI uses a 1–7 scale while the other four inventories are presented on 1–5 scales. Nevertheless, dominant dimension is computed by averaging all five raw means without normalization, giving TIPI a wider numerical range. A sensitivity check linearly converting TIPI to 1–5 happens to preserve all five dominant labels, but changes their means and does not resolve construct nonequivalence. The coefficient of variation also does not measure what the prose suggests. Although 100 runs are mentioned, Table 3 is reproduced, for the first three models, by taking a population standard deviation over only four questionnaire-level means per dimension, excluding TIPI. Thus GPT-4 Neuroticism at 20.49% or Llama 3.1 at 33.69% describes disagreement among instruments, not fluctuation across 100 responses. No run-level SDs or intervals are reported. The Llama 3.1 and 3.2 figures are not fully reproducible from Table 2; for Llama 3.2, the published Extraversion CV of 21.93% lies outside the 24.08–25.93% range compatible with the displayed rounding. The descriptive averages do show a pattern: under these prompts and items, raw Agreeableness, Conscientiousness, or Openness scores tend to be high and Neuroticism low. The declared dominant dimensions are Agreeableness for GPT-4, GPT-4o-mini, and Llama 3.2, Conscientiousness for Llama 3, and Openness for Llama 3.1. Without human norms, common calibration, uncertainty, statistical tests, or behavioral validation, these numbers do not establish personality profiles. Causal interpretations about fine-tuning and design objectives are not tested. The defensible conclusion is narrow: five aligned models produce different response patterns across inventories and families under a reworded self-report protocol. The work shows that instrument, response scale, and prompt matter; it does not establish psychological traits, psychometric reliability, removal of contamination, model-family personality, or behavioral prediction outside the questionnaire.
Research question
What Big Five scoring patterns do five LLMs produce when responding to reformulated versions of five inventories, and how much do the averages differ across dimensions, instruments, and model families?