PERSIST studies how much a model's response to the same personality item changes when the administration context changes. It combines the 44-item BFI and 27-item Short Dark Triad, 71 items and eight traits in total, with LLM-adapted versions, random order, paraphrases, five highlighted persona profiles, reasoning on or off, and conversation history. The method enumerates 29 models from eight families, spanning 1B to 671B parameters, although the abstract and introduction claim 25 open-source models spanning 1B to 685B; the paper does not reconcile these counts. Main experiments use temperature 0, a reported 250 runs, and one question per turn. Reasoning experiments use temperature 0.6, and Claude is limited to 70 runs. The central metric is the standard deviation of each 1–5 item response across runs, later aggregated by model and trait.
The descriptive pattern is clear. Question-level variability decreases with model size (Spearman, p<0.001), but does not vanish, and the authors display standard deviations above 0.3 even for models larger than 400B. Perplexity does not change with size (p=0.934), although it correlates moderately with question-level SD (rho=0.465). Mean scores become higher for Openness, Conscientiousness, Extraversion, and Agreeableness and lower for Neuroticism and Dark Triad traits as models scale. This can reflect alignment, extremity, and ceiling effects rather than a more prosocial personality. LLM-adapted instruments do not reduce SD relative to the originals (Wilcoxon p=0.286) and increase perplexity (p<0.001), so removing human-specific references is not sufficient to stabilize responses.
Interventions have heterogeneous effects. Greater reasoning effort raises SD in GPT-OSS; Qwen 3, Qwen 3 MoE, DeepSeek, and Claude show higher variability with reasoning in the aggregated analyses, while perplexity usually falls. Buddhist and teacher personas reduce SD relative to the assistant; the schizophrenia persona increases both SD and perplexity, but the antisocial persona does not significantly change SD (p=0.260), so the abstract overstates the result when it generalizes higher variability to plural 'misaligned personas.' Paraphrases increase variability only in the published group of four models at least 50B (p<0.01; smaller models p=0.244). Conversation history raises SD for the 19 models below 50B and lowers it for the four larger models; the change correlates negatively with size (rho=-0.512, p=0.0126).
The artifact exposes questionnaires, prompts, generation code, and analysis scripts, but it does not reproduce the results: the official repository contains no raw outputs, preprocessed CSVs, figures, or logs, and has no data release. The current code also treats model-by-question differences as independent observations in the paraphrase and history Wilcoxon tests even though Tables 3 and 4 label n as the number of models; the reasoning analysis likewise applies Mann–Whitney to question-level values although many comparisons are paired. This pseudoreplication can make p-values too optimistic. The generator interprets n_iter inclusively: n_iter=250 creates 251 conditions and n_iter=100 creates 101, while no run configurations or data are available to establish what was actually executed. The BFI bank contains 4,399 paraphrases, 4,377 unique strings, and only 99 entries for 'Can be moody,' not 100.
The evidence supports the conclusion that these 1–5 self-reports are sensitive to context, wording, and inference mode, and that a single score is not a stable measurement. It does not establish that models lack every form of behavioral consistency, that the selected SD predicts real safety failures, or that an architectural deficiency has been identified. Neither the original nor adapted questionnaires has psychometric validation for LLMs, no external behavior is tested, and SD >0.3 is not justified as a hazardous threshold. PERSIST is a useful stress test for administration fragility; its safety conclusions should be read as motivated hypotheses rather than empirical risk certification.