This EMNLP 2025 paper proposes MARK, a multi-call prompting pipeline for predicting individual World Values Survey answers and aggregate response distributions. It does not measure participant personality or stress. An LLM first assigns each demographic attribute a 0-100 stress score and summarizes the profile; it then infers dominant and auxiliary MBTI cognitive functions, completes tertiary and inferior functions by rule, generates one answer and weight under each function, and uses another call to review, reweight, and synthesize the final option. Stress, personality, cognitive trajectory, and explanation are therefore model-generated text rather than psychological observations. The paper uses WVS Wave 7 U.S. and China case studies, claims thirty demographic features, twenty clusters, 400 subjects and 102,800 question-answer pairs in the U.S., and twenty subjects/4,340 pairs in China. It compares GLM-4-air, GPT-4o-2024-08-06, and Doubao-1.5-pro at temperature .9 without published repetitions or uncertainty. On the U.S. sampled distribution, MARK reports 33.69%, 38.11%, and 46.98% accuracy. Against each model's strongest accuracy baseline, the actual gains are 6.71, 1.15, and 15.53 points, not a uniform 10%. Section 5.2 also conflicts with Table 2: against Demo.+Ideo.+Opiniontop3 the table yields +8.63, +5.04, and +15.53 points, not +4.3, +8.9, and +15.5. In China, MARK is not the most accurate GLM method: 31.58% versus 35.21% with Chinese prompts and 31.37% versus 35.60% with English prompts, despite the table caption's broad superiority claim. Artifact audit further changes the methodological interpretation. The extraction notebook does not cluster the thirty described demographics; it uses only SACSECVAL and RESEMAVAL, WVS secular-value and emancipative-value indices derived from survey responses. Group and representative selection are thus informed by the target domain. The same notebook hard-codes five clusters rather than twenty, selects at most twenty people within each, has no seed, calls drop_duplicates without keeping the result, and labels an SSE curve as silhouette analysis. Oracle personality is also not measured personality: personal_assign.py iterates Q1-Q258, uses each subject's actual answers, including target items, and accumulates LLM-generated personality scores. Table 17 compares a demographic predictor against this response-derived synthetic label; 70% role accuracy and 15/31% type accuracy do not validate MBTI. The robustness analysis in fact shows nearly overlapping curves for random, predicted, and oracle assignments, and the paper says personality similarity has minimal influence. This weakens causal attribution to personality. The released metrics also lack valid implementations. The notebook computes one minus SciPy Jensen-Shannon distance, not one minus the divergence it names. For EMD it passes probability-mass vectors as sample locations to wasserstein_distance; reversed binary distributions [.7,.3] and [.3,.7] therefore receive zero instead of .4 on ordered supports. The kappa script passes per-option count vectors to cohen_kappa_score as if they were two raters' labels, while the paper alternates Cohen and Fleiss. Isolated p-values are printed, including .6, but no test, sampling unit, code, predictions, repetitions, intervals, bootstrap, or multiplicity correction is released. The repository omits WVS data, samples, outputs, results, and requirements.txt; all three notebooks have zero executed cells and zero outputs. Paper, README, and code respectively name Gemini-2-pro, Gemini-2.5, and gemini-1.5-flash-exp-0827; augmentation code fails at print(a), appends each result twice in memory, and the 21 released JSON files contain 1,254 empty probability objects. The main runner also exposes a non-placeholder API credential that must be treated as compromised. The defensible conclusion is narrow: more structured calls and context can change and sometimes improve synthetic-survey predictions, especially with Doubao. The study does not establish personality, stress, cognition, faithful interpretability, significance, unseen-demographic generalization, or replacement of human surveys. A credible reference requires response-independent sampling, respondent-level metric recalculation, measured constructs or construct-neutral labels, held-out people/questions/groups, compute-matched ablations, repeated runs, stereotype auditing, and complete immutable artifacts.
Research question
Can a multistage pipeline inspired by MBTI type dynamics, which infers stress and cognitive functions from demographics, improve the zero-shot prediction of responses and distributions of the World Values Survey compared to demographic and opinion prompts?