PSYDIAL proposes a five-stage pipeline for generating personality-conditioned synthetic Korean dialogues: trait assignment, PersonaChat profile selection, GPT-4 generation, automatic filtering, and up to three regeneration rounds. Although the paper frames the task around Big Five personality, the experiment uses only one dimension, Extraversion, reduced to extrovert and introvert poles for each of two speakers. Of 4,000 initial dialogues, the filter rejects 1,051 for profile, 208 for personality, and one for style; regeneration yields 4,305 cumulative attempts and Table 4 reports 2,928 positives. Table 1, however, sums to 2,932 final dialogues, 715 E/E, 685 E/I, 763 I/E, and 769 I/I, an unexplained four-item discrepancy. Dialogues average 8.16 turns and 33.25 syllable-level tokens. Selected profiles expose stereotypical associations: travel, dancing, football, hiking, and swimming for extroversion, versus reading, video games, painting, being alone, and also hiking for introversion. A t-SNE plot suggests post-filter separation, but no separation metric or statistical test is reported. For downstream evaluation, the authors compare KoGPT2, KoBART, Kolang-T5, and KoDialoGPT-v0 under five settings: pretrained, personality-prompted, fine-tuned for three epochs on HuLiC, fine-tuned for three epochs on PSYDIAL, and PSYDIAL fine-tuning plus a personality prompt. PSYDIAL-trained models with prompting reach P-ACC of 0.881, 0.864, and 0.864 for KoGPT2, KoBART, and Kolang-T5, versus 0.653, 0.664, and 0.625 without the prompt; most BLEU and ROUGE scores also improve. This supports the narrower claim that training on labeled dialogues and restating the label in the prompt helps reproduce synthetic classes within this protocol. It does not establish broad personality, human realism, or psychometric validity. P-ACC comes from KLUE RoBERTa-base fine-tuned for five epochs on PSYDIAL, yet no train/test split or safeguard against leakage across generation, classification, and evaluation is described. There is also no human evaluation of Korean naturalness or personality; the authors themselves note English-like direct-translation expressions. The code audit further exposes circularity: the history used by GPT-4 to classify a dialogue still contains the explicit trait prompt, while the profile filter receives the selected profile rather than the dialogue. The official repository publishes three scripts and two profile CSV files, but not the final PSYDIAL corpus, fine-tuned models, splits, result artifacts, a dependency manifest, or a license. The paper offers a useful initial recipe for synthetic conversational data in a lower-resource setting, but the published evidence does not independently verify data quality, reproduce the full evaluation, or support generalization beyond binary Extraversion in Korean.
Research question
Can a pipeline based on GPT-4 prompting generate and filter Korean synthetic dialogues that express poles of extraversion, and does fine-tuning conversational models with those data improve automatic generation evaluated by overlap, perplexity, and a personality classifier?