The published article evaluates whether four LLMs can infer BFI-10 item scores and Big Five traits from semi-structured interviews. This review uses the definitive 25-page article and its official six-page supplement, retaining arXiv v1 only as a superseded source. The journal version materially expands the preprint: it adds GPT-5-Mini, adds Xinyu Li and Julina Maharjan to the author list, reorders the authors, and reports supplementary confidence intervals. Neither code nor participant data are available.
The dataset starts with 555 U.S. adults recruited across several studies of adjustment to adverse or transitional life events; 518 participants with complete and valid BFI-10 data enter the analyses. The conversations are not spontaneous everyday dialogue: they are scripted research interviews about daily activities, a personal challenge, emotion regulation, and recent positive and negative events. Text is lowercased, punctuation and English stopwords are removed, and repetitions or fillers are truncated by an unspecified procedure. This preprocessing removes some stylistic, function-word, negation, and conversational-dynamics signals that could matter for personality, and no raw-text ablation is reported.
GPT-4.1-Mini, GPT-5-Mini, Meta-LLaMA-3.3-70B-Instruct-Turbo, and DeepSeek-R1-Distill-70B predict ten BFI items with direct prompting and five traits with direct or chain-of-thought prompting. The reference is same-session BFI-10 self-report, not objective personality ground truth. In the 518 analyzed cases, alignment is weak: item correlations range from −0.18 to 0.27 and all trait correlations remain below 0.30. The best trait result is Conscientiousness for zero-shot GPT-4.1-Mini, r=0.25 (95% CI 0.167–0.329). Many intervals include zero, and some statistically non-zero effects are negative. MAE/RMSE often exceed one scale point on the 1–5 scale except in several Openness conditions. Models shift predictions toward moderate or high values and categorical agreement with self-report is minimal, with κ approximately −0.07 to 0.11.
Off-by-one rates from 0.57 to 1.00 do not contradict that failure. With only three ordered categories, a Moderate prediction is at most one category away from both Low and High; the metric penalizes only a jump between the extremes. Meta-LLaMA with CoT reaches 1.00 on three traits by concentrating predictions centrally while κ remains near zero. This review therefore does not present off-by-one as clinical or psychometric accuracy. There is also no mean-prediction baseline, human-reader baseline, simple statistical model, or demographic baseline; low MAE can result from regression to the center without valid individual inference.
Repeatability is tested only for GPT-4.1-Mini, on 50 participants and three runs, while input-segment order is randomized. It cannot support a claim that all four models are stable and it is not validity. The published version does not identify the exact within-model ICC specification; the supplement merely lists coefficients. Inter-model agreement excludes GPT-5-Mini and is low at item level (ICC 0.02–0.26); at trait level it is 0.54–0.76 under zero-shot and falls to 0.11–0.34 under CoT. The length ablation compares the first 100, first 1,000, and full transcript, confounding length with question order and content; methods call the units words while results call them tokens. Medium context produces the highest observed correlations, while full context increases several RMSE values.
The study usefully documents a limitation of current LLMs, but it does not isolate its cause or validate a personality-assessment system. BFI-10 has only two items per trait; the paper does not report reliability in this sample, the exact aggregation/reverse-scoring algorithm, multiplicity correction, or direct statistical comparisons between models or prompts. Cleaning, exclusions, model/API versions, dates, output parsing, and the length χ² test are insufficiently specified for reproduction. CoT rationales are shown as examples but their faithfulness is not evaluated, so they are not validated interpretability evidence. The defensible conclusion is narrow: under these interviews, prompts, models, and brief self-report reference, predictions show limited repeatability evidence but weak individual alignment and poor categorical agreement. The paper does not establish reliable personality inference, superiority to baselines, generalization to everyday conversations, or fitness for decisions about people.