This preprint compares two ways of profiling eight open-weight LLMs on ten Schwartz values and the five Big Five traits. The first administers PVQ-40, PVQ-21, BFI-44, and BFI-10 as Likert self-reports with two counterbalanced option orders. The second scores fixed candidate responses from the Value Portrait dataset by summed token log-probability: 520 query-response pairs from 104 real-world ShareGPT, LMSYS, Reddit, and Dear Abby queries, previously validated with 681 participants; 286 pairs are value-tagged and 228 trait-tagged when the human correlation reaches r ≥ 0.3. Models are Gemma 3 4B/27B, Qwen 2.5 7B/72B, Qwen 3 30B-A3B/235B-A22B, and GPT-OSS 20B/120B. Long and short versions of the same questionnaire type agree strongly in rank, mean Spearman 0.74 for PVQ and 0.77 for BFI, while agreement with generation-probability profiles is lower: 0.31/0.28 for values and 0.26/0.11 for traits. Established instruments also show much stronger item structure. In a construct-recognition task, seven models obtain mean F1 of 0.69–0.83 on established instruments and 0.09 on VP; five sentence encoders assign the correct construct to 77–81% of established items versus 11–26% of VP items. This is the strongest result: questionnaire wording makes the measured dimension highly transparent and can support construct-consistent answers without establishing a stable disposition. Across eight demographic personas, gender, age, political orientation, and education, model-averaged PVQ shifts resemble marginal ESS subgroup differences: mean cosine 0.60 for PVQ-40 and 0.47 for PVQ-21, with 62/80 and 55/80 matching signs. VP has mean cosine −0.03, aggregate cosine 0.007 with 95% bootstrap CI [−0.221, 0.236], and 40/80 matching signs, indistinguishable from 50%. Normalized shift magnitudes are 0.67/0.71 for the questionnaires, 0.20 for humans, and 0.37 for VP; this is a within-profile relative magnitude, not Cohen’s d. Interpretation must be narrower than the title. Value Portrait is not unconstrained generation: it scores five prewritten responses per query, sums token log-probabilities without length normalization, then averages by scenario and construct. The coverage check only ranks the highest VP candidate against ten free samples; its global median rank is four but model-specific medians range from one to eleven, and this does not validate all five candidates or every construct average. Compared profiles use different instruments, items, scales, and estimators. With only five traits Spearman is coarse; the eight models come from four related families; NDCG resolves Likert ties using NumPy argsort order; and the aggregate RQ2 row is the median of eight p-values rather than a combined test. In RQ4, 80 signs are not independent trials because persona pairs oppose each other, profiles are centered, and Schwartz values are correlated. Marginal ESS subgroup contrasts also do not isolate causal effects of age, education, politics, or country. The paper acknowledges that RQ1–RQ3 lack a matched human baseline and that its probability method is a controlled measurement between questionnaire response and open generation. Code and data are promised upon publication; as of 15 July 2026 the work is listed as under review and no study artifacts were located. The defensible conclusion is that human questionnaires can strongly measure a model’s ability to recognize item meaning and desirability and are insufficient evidence of stable traits or future behavior. The study does not yet establish that VP is a more accurate measure of real behavior or that its profiles predict unconstrained generation.
Research question
To what extent do the value and trait profiles obtained by asking Likert self-reports to LLMs coincide with profiles constructed from response probabilities to realistic queries, why do the questionnaires appear internally coherent, and do their shifts under demographic personas transfer to that controlled generative behavior?