PerDet-R1 is a nine-page paper published at the AAAI 2026 Bridge on LLM Reasoning and hosted officially on OpenReview; it should not be confused with an AAAI main-track paper. This review audits all nine pages of arXiv:2601.18582v1, its LaTeX source package, and the official OpenReview record last modified 8 March 2026. No repository, model, transformed dataset, reusable full prompts, outputs, or training/evaluation code was found. The source package contains only LaTeX, bibliography/style files, and four figures.
The system takes up to 50 posts per user, capped at 128 tokens each, and returns the three most likely MBTI types. It starts from Qwen2.5-7B-Instruct. During SFT, Qwen-plus generates a <think> rationale and top-3 list; examples are retained only when the recorded MBTI label occurs in that top-3, until 1,000 are selected. During GRPO, the remainder of the dataset is used as training data and each group contains 16 generations. Reward adds NDCG over candidates to another dimension-similarity score applied to the first candidate.
Providing ordered alternatives can be useful, but there is no ranking ground truth here. Each user has one four-letter MBTI label. Relevance for the other fifteen types is authored by counting character matches with that label: E/I receives weight 1.1, S/N 1.2, T/F 1.3, and J/P 1.4 when epsilon=.1. There are no human data saying which type is second or third, no trait intensities, instrument probabilities, or psychometric distances. Increasing weight by letter position is not psychologically justified.
Total reward is dominated by the first answer. NDCG is approximately bounded from 0 to 1, whereas top-1 Dimension Similarity ranges from 0 to 5 and reuses the same relevance signal already used by NDCG. The objective is therefore primarily a softened top-1 Hamming classifier, not preference learning. The IDCG equation prints an unindexed s_sorted and does not say whether the ideal is computed over the three generated candidates or all sixteen types. GRPO normalization divides by group reward standard deviation without describing the all-equal-reward case.
The paper claims ranking models a continuous spectrum and eliminates a “categorical fallacy,” yet output remains a list of discrete categories and primary evaluation takes the first item as a class. It does not estimate continuous position along E–I, S–N, T–F, or J–P. Even official MBTI materials distinguish type theory, which sorts preferences, from trait theory, which measures amounts continuously; ordering types by shared letters does not turn a typology into a continuous trait.
The most serious methodological omission is the split. No train/validation/test protocol, sizes, stratification, seed, checkpoint selection, or held-out test is reported. The text literally says the RL stage uses the original datasets excluding the 1,000 SFT samples. If that means all remaining records, no test remains described. The SOTA table cannot be interpreted as generalization without an undocumented protocol. It is also unclear whether 1,000 SFT examples are drawn per dataset or in total, how they are selected, what fraction Qwen-plus rejects, or their class distribution.
The benchmarks contain substantial shortcuts. Kaggle has 8,675 profiles with 50 recent posts from PersonalityCafe, a forum devoted to MBTI discussion. Exact type strings are masked, but the authors acknowledge that abbreviations such as NS, ExFJ, and xNTP remain and improve performance. Terms such as “introvert,” cognitive-function abbreviations such as Ni/Te, and forum context can remain too. PANDORA labels are predominantly self-reported in Reddit flairs; its original paper warns that the sample is not representative of Reddit or the general population and reports severe imbalance: 8,024 Intuitive versus 1,030 Sensing and 7,134 Introverted versus 1,920 Extraverted among 9,054 users with complete dimensions. PerDet-R1 says it uses 9,067 users without reconciling this with the original 9,084 MBTI reports or explaining exclusions.
Table 1 does show the highest point estimates: Kaggle 80.57 binary macro-F1 and 41.34 multiclass F1; PANDORA 66.10 and 35.08. But narrative margins use the wrong comparators. On Kaggle multiclass, Qwen-plus reaches 38.63 and is a stronger baseline than ETM at 32.55, so the SOTA margin is 2.71 points, not 8.79. On PANDORA multiclass, TAE at 30.22 exceeds ETM at 30.09, making the margin 4.86 rather than 4.99. On PANDORA binary, 66.10−65.77 is .33 points, not the reported .83. Only Kaggle binary is correct: 80.57−77.79=2.78.
The paper repeatedly uses percent signs for absolute score subtraction. Ablation “drops” of 8.95, 10.22, 20.49, 24.85, 4.93, 7.39, 4.13, 2.24, and .60 are table-point differences, not relative percentages. It also calls differences “significant” without tests, intervals, seeds, means, standard deviations, or distributions. Figure 4 is a smoothed training-reward curve from one run, not held-out performance; the red dip near step 800 recovers and cannot alone establish stability or freedom from reward hacking.
Although framed as ranking, the main table reports no PANDORA NDCG and compares NDCG against no ranking baseline. NDCG appears only in internal Kaggle ablations. SOTA results convert the top item into binary/multiclass predictions. Multiclass “F1-score” is not defined as macro, micro, or weighted; for single-label multiclass data, micro-F1 equals accuracy, making the definition essential. There are no per-type scores, confusion matrices, or minority-class analysis.
SFT rejection retains rationales only when Qwen-plus places gold in the top three. This introduces survivorship bias toward users, classes, and examples the teacher already solves. The paper says filtering ensures “highly readable” reasoning, but it only checks label presence; readability, explanation correctness, and faithfulness are not evaluated. Better F1 after training on traces does not demonstrate better reasoning: text may be post-hoc rationalization, and there is no direct-answer control at matched compute.
The claim of eliminating expert dependency is also unsupported. The teacher prompt requires “official interpretations of 16 MBTI personality types,” output format and top-k are hand-designed, and reward manually encodes letter similarity, dimension weights, and duplicated top-1 priority. There is no cost/latency comparison with prompt baselines or evidence of real-time deployment. Qwen-plus temperature=1 is incorrectly described as “mostly deterministic.”
Reproducibility is low: learning rate, scheduler, LLM optimizer, LoRA alpha/dropout/target modules, total context and truncation order, clipping epsilon, KL beta, GPUs, precision, GRPO framework, versions, checkpoint selection, seeds, and cost are absent. No model or artifact release exists. Arithmetic and methodological claims cannot be checked against outputs.
The defensible conclusion is narrow: under an incompletely specified protocol, a Qwen2.5-7B model post-trained on filtered traces and a label-similarity reward obtains higher point estimates than the listed baselines on two self-reported MBTI datasets with shortcuts. It does not establish a continuous spectrum, real psychometric ranking, faithful reasoning, human personality, clinical validity, mental-health screening, out-of-forum generalization, or even held-out performance until the split and pipeline are released.