OmniBehavior proposes a behavior-simulation benchmark built from real Kuaishou traces. The paper describes 200 users, five scenarios, 22 action types, and 2.12 million interactions from September through November 2025. Each profile combines demographic and household attributes with histories that may include exact timestamps, video text, OCR, ASR, searches, products, advertisements, and customer-service conversations. Users are selected with K-means by taking the person nearest each of 200 centroids: this broadens diversity coverage but does not produce a population-representative sample or support prevalence estimates. The main task supplies a profile, history, and current context to predict binary actions, video duration, or the next customer-service reply. The primary setup uses September for history and October-November for testing, with a claimed 30 test actions per user and 6,000 cases. Eleven models are evaluated. Claude Opus 4.5 scores 44.55 on the published ranking, followed by GLM-4.7 at 41.46 and Claude Sonnet 4.5 at 40.49; Qwen3-235B scores 32.11. The overall score, however, is an unweighted mean of six heterogeneous components: binary-scene F1, 100 minus NMAE for duration, and a Claude-Sonnet-4.5 text score multiplied by ten. No intervals or tests accompany rank differences, and Claude-Sonnet-4.5 is simultaneously a competitor, text judge, and sentiment/style annotator without human validation. For memory, Qwen3-235B scores 24.27 with summarization, 21.13 with truncation, and 20.38 with RAG; these are averages across unlike scenes and do not support a general conclusion about all memory systems. Across 66 histories longer than 128k tokens, increasing context does not improve performance consistently. The paper calls 180 pre-conversion trajectories inferred by Claude and manually reviewed causal chains; because they come from observational logs without an intervention or counterfactual, they are retrospective attributions rather than causal effects. It also compares daily Jaccard drift in OmniBehavior, .6311, with LoCoMo, .1698, but platform, tasks, and generation differ and do not isolate a general real-versus-synthetic property. Its hyperactivity, utopian-bias, and personality-homogenization labels depend on balanced sampling, Claude annotations, and action-rate vectors; the last measures behavioral similarity, not validated personality. Artifact inconsistencies further constrain interpretation. Dividing 2.12 million interactions by 200 users gives 10,600 per user, whereas the reported 8,143 average implies 1,628,600, an unexplained difference of 491,400. In a deterministic audit of 20 pairs from the public snapshot, English JSON files contain 1,324 actions and Chinese files 2,688; only five pairs match, with the difference concentrated in video events, 355 versus 1,719. This establishes release drift in the audited sample, not a corpus-wide mismatch rate. The code requires at least 60 actions to construct 30 cases, while audited released users fall below that threshold, so the current artifact does not guarantee 200 users or 6,000 tasks; it also lacks reproducible configuration for all eleven models, outputs, judgments, and results. The dataset is public and ungated. Although direct identifiers are replaced, precise timestamps, demographics, and rare content remain longitudinal quasi-identifiers; consent, privacy evaluation, withdrawal, takedown contact, and anonymization performance are undocumented. The work contributes a rich industrial resource and inspectable code, but its findings should be read as results on a curated benchmark and an arXiv v2 preprint, not as representative simulation, causal evidence, or validated human personality measurement.
Research question
To what extent can current LLMs simulate heterogeneous human actions from long, multi-scenario histories, and what structural differences appear between their predictions and real traces?