This work attempts to infer human traits, not machine personality. Amazon Mechanical Turk participants completed BFI-44, IOS, KISS-18 and ATQ measures and held three conversations with one of three systems: a pipeline task system, task-oriented SimpleTOD or open-domain BlenderBot. After filtering, 179, 199 and 186 participants remained. BERT-base-uncased receives only user messages and learns 25 separate targets.
“Personality prediction” here means classifying whether a score is above or below the training-plus-validation median. It does not estimate a continuous score or normative level. The published rule also leaves exact median ties undefined, important because questionnaire totals are discrete. Thresholds change across splits, so .64 balanced accuracy describes a relative, moving label rather than a stable individual assessment.
The strongest results occur in open-domain dialogue: .71 Conscientiousness, .68 Agreeableness, .60 Neuroticism, .61 IOS and .60-.68 for five temperament/social facets. Extraversion reaches .64 in both open-domain and pipeline task dialogue. Many of the 25 outcomes remain near the .50 baseline. The only comparator is a majority classifier whose balanced accuracy is .50 by construction; length, vocabulary, TF-IDF and linear baselines are absent, so BERT's added value is unknown.
The open-domain advantage is not causal. Each condition uses different people, with no reported random assignment or cohort demographics. Domain, architecture, difficulty, turn limit, content, system persona, quality and success rate change together. Open-domain dialogue also contains more user vocabulary (90.2 terms per dialogue versus 70.8 and 58.1) and higher lexical diversity. The comparison is between complete packages and populations.
Cross-domain performance is often near chance, useful evidence of distribution shift. But only this BERT and three historical corpora are tested without domain adaptation; the study cannot show that open-domain data can never aid task prediction. Nor does it establish that one round is sufficient or task success irrelevant: these are absence-of-significance claims without equivalence tests or power. The proposed SimpleTOD free-expression mechanism rests on success rates and scatterplots, not causal analysis.
Statistical inference is not reproducible: the article names a “non-parametric t-test”, a contradictory label, and compares overlapping cross-validation estimates without identifying the test statistic. Questionnaire scoring, reverse coding, reliability and distributions are also omitted. Data and code are available only “upon reasonable request”; no public release was found through Nature, PMC, the author page or author GitHub.
The faithful conclusion is modest: in these AMT samples and historical systems, some relative trait groups leave textual signals recoverable by BERT, especially in unconstrained dialogue. Continuous validity, operational utility, fairness, personalization benefit and generalization to current LLMs are not validated. Balanced accuracy of .60-.71 still implies substantial error, and the study never deploys adaptive decisions to test benefit or harm.