Population-Aligned Persona Generation for LLM-based Social Simulation

Research question

Can a set of narrative personas generated from human digital fingerprints be constructed that, by conditioning several LLMs, better reproduces global and target-group psychometric distributions than existing public persona sets?

Method

The method aggregates publications by author from the Blog Authorship Corpus, cleans and filters the texts, generates a persona of up to one hundred words with a 70B Llama 3 variant, and scores it with Qwen2.5-72B on hallucination, coverage, conciseness, relevance, and quality. It retains profiles with a global score above 8 and all dimensions above 7. For global alignment, a frozen LLM responds to the 50 IPIP Big Five items conditioned by each persona; densities are estimated via KDE, the top 70% is resampled by importance weights, and the selection is refined with entropic optimal transport and Sinkhorn, until obtaining 5,000 personas. For specific groups, Qwen2.5-72B generates two queries per persona and filters negatives; Qwen3-Embedding-0.6B is trained contrastively to retrieve profiles, which Qwen then rewrites following instructions from YRBSS or WVS. Three base models and six public or synthetic sets are compared via AMW, Frechet distance, sliced Wasserstein, MMD, and absolute error between correlation matrices. The review recalculated averages and percentages from the four tables, contrasted the corpus statistics with its primary source, examined the prompts, the two theoretical proofs, the privacy section, and the twenty rendered pages.

Sample: The source of profiles is the Blog Authorship Corpus: its primary documentation describes 681,288 publications from 19,320 bloggers collected in 2004. Table 10 of the preprint reports 681,000 raw texts, 500,000 after preprocessing, 368,000 after quality control, 41,607 drafts per user, and 30,738 final personas; global alignment selects 5,000. The IPIP reference contains 1,015,341 responses and the article declares 224 countries or regions. For additional evaluation, Table 7 declares CFCS with 15,034 responses from 144 countries, FBPS with 41,841 from 169, Duckworth with 4,270 from 111, YRBSS with 20,103 responses and 105 items, and complete WVS with 97,221 responses and 294 items; the regional experiments generate 1,000 personas per scenario. The manuscript offers other incompatible counts, more than 30,000 users, more than 100,000 and more than 160,000 personas, which cannot be reconciled with the primary source or with Table 10.

Findings

• Table 1 reports that Resample achieves the lowest global average among the compared methods: 0.1715.
• In Qwen2.5-72B, Resample obtains AMW 0.1527, FD 0.2628, SW 0.1272, and MMD 0.0840.
• In Llama-3-70B, Resample obtains 0.2013, 0.4036, 0.1955, and 0.1937 on those four metrics.
• In Phi-4, Resample obtains 0.0889, 0.1911, 0.1214, and 0.0353.
• Raw and RandomSelect produce nearly identical averages, 0.2580 and 0.2577, which supports that the main change comes from distributional selection and not from subset size.
• The improvement percentages of 49.8%, 37.9%, and 45.1% announced for Table 1 are not obtained from a single and explicit rule applied to the visible averages.
• Models without persona improve when temperature increases, but remain far behind persona-conditioned methods in Table 1.
• Figure 3 visually confirms that temperature adds dispersion but does not by itself cover the human distribution of extraversion and emotional stability.
• Figure 3 also shows that no synthetic set perfectly matches the human cloud, despite the full coverage language used in the text.
• Table 2 reports for Resample a real average of 0.2085 on CFCS, FBPS, and Duckworth.
• The text declares 0.2279 for the same average of Table 2, in contradiction with the table.
• Compared to the Bavard average of 0.3344, 0.2085 represents an approximate reduction of 37.6%, not the declared 32%.
• In FBPS, Table 2 reports FD 0.4004, while the text states 0.4220.
• The text calls the fourth metric of Table 2 MED, but the table and the method identify it as MMD.
• Resample presents the minimum value of all cells in Table 2 among the compared methods.
• The transfer of Table 2 is to distinct questionnaires within the same ecosystem of online psychometric responses, not to new representative populations.
• Table 3 reports for Resample MAEcorr 0.4108 in Big Five, 0.2270 in CFCS, 0.3607 in FBPS, and 0.4255 in Duckworth.
• The correct average of that row is 0.3560, as shown in Table 3.
• The text declares 0.3651 and an improvement of 16.2% over Bavard; the tabulated values imply approximately 18.3%.
• MAEcorr compares aggregated correlation matrices between human and synthetic populations; it does not measure longitudinal consistency within each individual.
• Table 4 reports 4.5329 for the trained contrastive retriever, the best average among the shown rows.
• The declared 19.1% improvement over AlignX is compatible, by rounding, with 4.5329 versus 5.6136.
• The 8.9% improvement over Resample cannot be verified because Table 4 does not include a Resample row.
• The untrained retriever obtains 4.8702 and also outperforms the shown external sets.
• The term untrained describes the pretrained embedding model without the contrastive fine-tuning of the article, not a model without prior training.
• The group evaluation uses YRBSS and three WVS regions: Hong Kong, the United States, and Germany.
• The group prompts require that the Hong Kong profile represent the vision, social trust, and cultural identity of the general population.
• The German prompt prescribes typical opinions about family, work-life balance, and social welfare.
• The U.S. prompt prescribes mainstream perspectives on national identity, civic participation, and social justice.
• The YRBSS prompt requires including diet, activity, sexual behavior, substance use, mental health, and safety.
• These instructions provide the generator with the topics and stereotypes that are later compared with the target surveys.
• The documented seed chain in Table 10 ends in 30,738 high-quality personas.
• That total contradicts the claim of more than 160,000 personas in Section 3.1 and that of more than 100,000 in the appendix.
• The original Blog Authorship Corpus documentation reports 19,320 bloggers, not more than 30,000.
• The manuscript speaks in the plural of three open datasets or of each source, but only identifies the Blog Authorship Corpus for mining.
• The original corpus contains blogs from 2004 and three discontinuous age bands, so it does not represent the contemporary population.
• The work uses an online volunteer sample from OpenPsychometrics as a proxy for the global human population.
• The appendix itself recognizes the underrepresentation of rural, elderly, offline, and under-resourced communities.
• The safety and quality filter eliminates approximately 30% of profiles and may reduce precisely the adverse experiences relevant for social simulations.
• The privacy section reports that GPT-4o flags 0.69% of personas for rewriting and 99.31% as no PII according to its narrow taxonomy.
• The detector only considers full name, postal address, email, phone, and bank card or account.
• The negative label of the detector is a model output, not a proof of anonymity or of absence of reidentification.
• Five annotators review the outputs according to the manuscript, but no protocol, agreement, training, compensation, or demographic data are reported.
• Names of public figures are deliberately excluded from PII.
• The complete prompts for cleaning, generation, evaluation, group, and privacy are included, which improves conceptual inspection.
• The appendix identifies checkpoints and several hyperparameters: KDE 0.20, retention 70%, epsilon 0.08, batches of 10,000, and 250 Sinkhorn iterations.
• The version of Llama used to generate seeds changes between Llama-3-70B, Llama-3.3-70B, and their checkpoint names.
• The manuscript retains Trovato et al. headers and editorial dates of 2007-2009 from the ACM template, a sign that it is not a final editorial version.
• No peer-reviewed publication or specific repository of code, personas, or results associated with the work was found until July 15, 2026.
• The evidence supports a reduction in response distances under the chosen protocol, not the accuracy of a complete social simulation.

Limitations

• The document is a preprint and no peer-reviewed version was found.
• The residual template headers and dates indicate incomplete editorial preparation.
• The code for the data chain, sampling, transport, or evaluation is not published.
• The final set of 30,738 personas is not published.
• The 5,000 selected personas are not published.
• The personas of the four target groups are not published.
• The raw LLM responses to the questionnaires are not published.
• The KDE, importance, or optimal transport weights are not published.
• An executable artifact to reproduce the tables is not published.
• A license for the derived data is not reported.
• The corpus source limits its use to non-commercial research and the manuscript does not discuss the implications for releasing derivatives.
• The provenance of the figure of more than 160,000 personas is not explained.
• The provenance of the figure of more than 100,000 personas is not explained.
• Those figures are not reconciled with the 30,738 personas in Table 10.
• The more than 30,000 users in the appendix are not reconciled with the 19,320 authors in the primary source.
• The text alludes to three open datasets, but documents only one.
• The plural grammar of several sources hides that the final table contains only blogs.
• The Blog Authorship Corpus was collected in 2004 and temporal drift is not analyzed.
• The corpus contains only Blogger authors and not a probabilistic sample.
• The corpus is restricted to English.
• The ages of the corpus are concentrated in 13-17, 23-27, and 33-47 years.
• It does not comparably include childhood, 18-22, 28-32, or over 47 years.
• The binary sex balance of the corpus does not equate to gender diversity.
• The age, gender, industry, and zodiac labels were self-reported and their accuracy is not validated.
• The geographic representativeness of the bloggers is not studied.
• Socioeconomic differences in blog access in 2004 are not studied.
• Consent of the authors to generate psychological profiles from their texts is not reported.
• Ethical review, IRB, or exemption is not reported.
• Whether the aggregation of publications allows reidentification of their authors is not discussed.
• Removing direct identifiers does not eliminate unique biographical combinations.
• The PII detector omits partial names, aliases, and usernames.
• The detector omits employers, educational institutions, and rare occupations.
• The detector omits dates, exact ages, and combined locations.
• The detector omits singular life events and family relationships.
• The detector omits stylometric fingerprints.
• The detector allows names of public figures without justifying the risk of attributed profiles.
• GPT-4o is not identified by snapshot or date in PII detection.
• Sensitivity, specificity, or test set of the detector are not reported.
• The 99.31% negative is not validated against an independent human standard.
• How many discrepancies the five reviewers detected is not reported.
• Inter-annotator agreement is not reported.
• How disagreements were resolved is not reported.
• Whether the reviewers saw the original texts is not reported.
• Compensation or working conditions of the reviewers are not reported.
• Removing content considered harmful may erase experiences of violence, harassment, or discrimination.
• That filtering introduces a positivity bias recognized by the authors.
• The positivity bias is especially problematic when evaluating YRBSS, where mental health, violence, and consumption are target variables.
• The LLM that generates personas and the one that evaluates their quality belong to an automated evaluation chain without comprehensive human validation.
• The fidelity of the 30,738 personas to their source publications is not manually validated.
• The quality thresholds of 8 and 7 are arbitrary and no calibration is provided.
• The distribution of scores or error of the critic is not reported.
• Sensitivity to the generation or critique prompts is not studied.
• Sensitivity to the persona-generating model is not studied.
• The exact version of Llama for seed generation is contradictory.
• Qwen2.5-72B is used both for several generations and for filtering, creating model circularity.
• Which LLM responds to the questionnaire during each alignment phase is not clearly specified.
• Profiles are evaluated through LLM responses and not through observed properties of their human authors.
• Alignment may select personas that induce desired responses without being faithful human descriptions.
• IPIP OpenPsychometrics is an online convenience sample, not a global probabilistic survey.
• Counting countries or regions does not correct the imbalance in sizes between countries.
• Survey weights or demographic post-stratification are not applied to IPIP.
• The composition by country, age, sex, education, or language of the IPIP reference used is not reported.
• The manuscript alternates between 223 and 224 countries or regions.
• Measurement invariance of IPIP across cultures or languages is not demonstrated.
• Responses from different translations may not be directly comparable.
• CFCS, FBPS, and Duckworth also come from voluntary online psychometric responses.
• Therefore, out-of-domain evaluation changes instrument but does not guarantee change of population or collection mechanism.
• The item numbers of FBPS are internally incompatible: 25 in the description and 76 in Table 7.
• The item numbers of Duckworth are internally incompatible: 12 in the description and 50 in Table 7.
• Which versions or adaptations of CFCS, FBPS, and Duckworth were used is not explained.
• Psychometric properties of the scales in the samples used are not provided.
• Measurement invariance between humans and LLM-generated responses is not demonstrated.
• Aquiescence, social desirability, or response styles of the LLMs are not controlled.
• The order of items or whether it was randomized is not reported.
• How many replicas were generated per persona and model is not reported.
• Random seeds are not reported.
• Standard deviations are not reported.
• Confidence intervals are not reported.
• Hypothesis tests for the use of significantly are not performed.
• Correction for multiple comparisons is not applied.
• Variability between sampling or Sinkhorn runs is not reported.
• Sensitivity to the KDE bandwidth 0.20 is not analyzed.
• Sensitivity to retaining 70% of candidates is not analyzed.
• Sensitivity to epsilon 0.08 is not analyzed.
• Sensitivity to the per-item weights of the transport cost is not analyzed.
• The final size of 5,000 personas is not justified.
• Comparison against classical demographic resampling or raking is not performed.
• Comparison against an equivalent probabilistic human sample is not performed.
• In-domain evaluation reuses IPIP as the alignment target and as the success criterion.
• This reuse partly measures direct optimization toward the target.
• The four distributional metrics are not independent and are averaged without justifying scales or weights.
• The improvement percentages of Table 1 are not accompanied by a reproducible formula.
• The average of Table 2 in the text contradicts the table.
• The improvement percentage of Table 2 contradicts the tabulated values.
• The FD of FBPS in the text contradicts Table 2.
• The name MED contradicts MMD.
• The average of Table 3 in the text contradicts the table.
• The improvement percentage of Table 3 contradicts the tabulated values.
• Aggregated MAEcorr is presented as individual consistency without measuring within-person stability.
• The improvement over Resample in Table 4 cannot be verified because that row is absent.
• Whether untrained retains the pretrained Qwen checkpoint is not explained, although the context suggests it does.
• WVS regions are reduced to one representative country or territory per block.
• Hong Kong does not represent all of East Asia.
• Germany does not represent all of Western Europe.
• The United States does not represent all of North America.
• WVS survey weights are not used in the described comparison.
• YRBSS survey weights are not used in the described comparison.
• The regional prompts directly introduce expected values and attitudes.
• The word mainstream in the U.S. prompt reduces political and social diversity.
• The typical views instruction of the German prompt turns generator stereotypes into a target.
• The representative of the general populace instruction for Hong Kong does not define a measurable distribution.
• The YRBSS prompt requires completing sensitive attributes even if they are not in the seed.
• Rewrites may fabricate sexual behavior, consumption, mental health, or safety experiences.
• Stereotypical harm or stigmatization in group profiles is not measured.
• Protected subgroups within each region are not evaluated.
• The content of generated personas is not compared with human interviews.
• Individual fidelity between seed and regional rewrite is not evaluated.
• Whether the retriever memorizes expressions from synthetic queries is not evaluated.
• Training positives and negatives are generated or filtered by Qwen, not by independent human relevance.
• The final size of the contrastive set is not reported.
• An independent partition of personas to train and evaluate the retriever is not reported.
• Generalization to groups formulated outside the four prompts is not demonstrated.
• No real multi-agent dynamic is evaluated.
• Conversations between personas are not evaluated.
• Temporal or event predictions are not evaluated.
• Intervention or public policy decisions are not evaluated.
• A simulated output is not compared with observed social outcomes outside the questionnaires.
• Distributional matching of responses does not imply social causality.
• Population matching does not guarantee that each persona is coherent or true.
• Matching of marginals and correlations does not guarantee higher-order dependence structures.
• Stability of the same persona across base models is not evaluated.
• Stability to questionnaire paraphrases is not evaluated.
• Longitudinal stability is not evaluated.
• Total computational cost is not reported.
• Total number of calls or tokens is not reported.
• Execution time, hardware, energy, or economic cost is not reported.
• Construction requires generating and scoring hundreds of thousands of texts and responding to millions of items, which limits replication.
• The proof of Theorem 1 equates unbiasedness of weights with exact equality of distributions.
• An importance distribution with finite support need not exactly match the empirical human distribution.
• Theorem 1 states that the sum of exact weights is Θ(1), although without prior normalization it normally scales with N.
• Theorem 1 applies a Wasserstein bound with square root of TV but retains the same error order without the root.
• The declared empirical Wasserstein rate omits the usual dependence on dimension.
• The conditions of support, continuity, minimum density, and regularity necessary for the KDE bounds are not enumerated.
• Theorem 2 states that all entries of a coupling are at least 1/(N times M), but feasible couplings may contain zeros.
• The proof itself obtains epsilon times log(N times M), which is not O(epsilon) uniformly when N and M vary.
• The appendix theorem uses an unweighted Euclidean cost while the main method defines an item-weighted cost.
• The transport guarantee refers to the cost between response vectors, not to human fidelity of the biographies.
• The two proofs do not justify the phrase provably recovers the target distribution.
• There is no preregistration.
• No independent replication is reported.
• The possibility of negative results or model failures is not discussed.
• A data sheet or model card for the derived personas is not provided.
• Governance of access, withdrawal, or deletion for the corpus authors is not established.

What the study does not establish

• It does not demonstrate that the final set represents the world population.
• It does not demonstrate national or regional representativeness of IPIP, WVS, or YRBSS without weighting and explicit sampling design.
• It does not demonstrate that the generated profiles faithfully describe the original bloggers.
• It does not demonstrate that the bloggers consented to psychometric inference or persona generation.
• It does not demonstrate that 99.31% of profiles are anonymized or free from reidentification risk.
• It does not demonstrate that LLM filtering eliminates hallucinations.
• It does not demonstrate that generated psychometric scores equate to human responses.
• It does not demonstrate psychometric invariance across countries, languages, humans, and LLMs.
• It does not demonstrate that minimizing AMW, FD, SW, or MMD produces individually valid personas.
• It does not demonstrate individual consistency through MAEcorr, because it compares population correlations.
• It does not demonstrate generalization outside related online psychometric questionnaires.
• It does not demonstrate that regional prompts discover culture rather than impose stereotypes.
• It does not demonstrate that Hong Kong, Germany, or the United States represent their complete WVS regions.
• It does not demonstrate that YRBSS personas correspond to real adolescents.
• It does not demonstrate that fabricating sensitive attributes is ethical or safe.
• It does not demonstrate a statistically significant improvement because it reports no uncertainty or replicas.
• It does not demonstrate that the announced percentages are reproducible from the tables.
• It does not demonstrate the validity of Theorem 1.
• It does not demonstrate a uniform O(epsilon) bound in Theorem 2.
• It does not demonstrate exact recovery of the target distribution.
• It does not demonstrate realistic behavior, interaction, or social dynamics.
• It does not demonstrate predictive capacity regarding elections, health, conflict, or observed values.
• It does not demonstrate utility or safety for public policy decisions.
• It does not demonstrate reproducibility without code, derived data, weights, and raw outputs.
• It does not justify deploying personas as substitutes for human participants.

Traceability

Scope: Full text

Version: arXiv 2509.10127v2, 4 Oct 2025; open 20-page preprint

Consulted source: https://arxiv.org/pdf/2509.10127v2

Review: Codex full-text, bilingual-fidelity, visual, bibliographic, dataset-provenance, psychometric-validity, metric-recalculation, theoretical-proof, privacy, ethics, internal-consistency and reproducibility audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

• Meta-Llama-3-70B-Instruct
• Meta-Llama-3.3-70B-Instruct
• Qwen2.5-72B-Instruct
• Qwen3-Embedding-0.6B
• Phi-4
• GPT-4o

Instruments and metrics

• IPIP Big Five 50-item inventory
• Consideration of Future Consequences Scale (CFCS)
• Free Will and Punishment and Retribution Scale (FBPS)
• Duckworth Grit Scale
• World Values Survey (WVS)
• Youth Risk Behavior Surveillance System (YRBSS)
• Averaged Monotonic Wasserstein (AMW)
• Fréchet Distance (FD)
• Sliced Wasserstein Distance (SW)
• Maximum Mean Discrepancy (MMD)
• MAEcorr between human and synthetic correlation matrices

Data used

• Blog Authorship Corpus
• OpenPsychometrics IPIP Big Five responses
• OpenPsychometrics CFCS responses
• OpenPsychometrics FBPS responses
• OpenPsychometrics Duckworth responses
• World Values Survey responses
• Youth Risk Behavior Survey responses
• Tulu-3-Persona
• Bavard
• Google Synthetic Personas
• AlignX
• Nvidia Nemotron personas
• PersonalHub
• SyncP personas

Evidence and location

• Bibliographic identity and editorial status: arXiv 2509.10127v2, submitted 12 Sep 2025 and revised 4 Oct 2025; Microsoft Research and DBLP list it as arXiv/CoRR; no peer-reviewed venue found, checked 15 Jul 2026
• Open full text inspected: .cache/editorial-sources/article-086/source.pdf; 20 pages; sha256 1ae6da2319faaf7ea959c44c06fe1c6287033585cd91fd9f5c40d8bb00a2cb35
• Author order: arXiv v2 metadata and title page: Zhengyu Hu, Zheyuan Xiao, Max Xiong, Yuxuan Lei, Tianfu Wang, Jianxun Lian, Kaize Ding, Ziang Xiao, Nicholas Jing Yuan, Xing Xie
• Seed generation chain: Sections 3.1 and Appendix F, pp. 3 and 17-20; Listings 13-16
• Final seed statistics: Table 10, p. 20: 681,000 raw; 500,000 preprocessed; 368,000 raw-QC; 41,607 persona generation; 30,738 persona-QC
• Primary corpus statistics: Official Hugging Face Blog Authorship Corpus dataset card: 681,288 posts, 19,320 bloggers, collected August 2004, over 140 million words, non-commercial research use
• Scale and source contradictions: Section 3.1 p. 3 says over 160,000; Appendix B.1 p. 11 says over 30,000 users; Appendix F p. 17 says over 19,000 and over 100,000; Table 10 says 30,738
• Global alignment: Section 3.2, pp. 3-5; equations 1-9; KDE importance sampling, 70% candidate retention, entropic OT and final sample of 5,000
• Hyperparameters: Appendix B.1, pp. 11-12: Gaussian KDE bandwidth 0.20, epsilon 0.08 scaled by median cost, batch size 10,000 and 250 Sinkhorn iterations
• Group-specific construction: Section 3.3, pp. 5-6; Appendix B.6, pp. 14-15; Listings 4-7
• Stereotyped regional prompts: Listings 5-7, pp. 14-15: representative general-populace Hong Kong views, typical German views, and mainstream U.S. perspectives
• Sensitive YRBSS prompt: Listing 4, p. 14: requires diet, physical activity, sexual behavior, substance use, mental health and safety experiences
• Models and checkpoints: Introduction, Sections 3.1-4 and Appendix B.1/F: Qwen2.5-72B, Qwen3-Embedding-0.6B, Llama-3-70B, Llama-3.3-70B, Phi-4 and GPT-4o; inconsistent Llama naming
• In-domain results: Table 1 and Section 4.1, pp. 6-7; Resample averages 0.1715 and Raw/RandomSelect 0.2580/0.2577
• Out-of-domain results and contradictions: Table 2 and adjacent prose, p. 7: table average 0.2085 versus prose 0.2279; FBPS FD 0.4004 versus 0.4220; table MMD versus prose MED
• Aggregated correlations and contradiction: Table 3 and Section 4.2, p. 7: row average 0.3560 versus prose 0.3651; metric compares Pearson correlation matrices
• Group results: Table 4, p. 8: EM w train 4.5329, EM w/o train 4.8702, AlignX 5.6136; Resample comparison claimed but row absent
• Incomplete visual coverage: Figure 3, pp. 7-8: synthetic clouds remain distinguishable from the human distribution despite improved coverage
• Samples and instruments: Appendix B.2 and Table 7, pp. 12-13; includes incompatible FBPS and Duckworth item counts between prose and table
• Privacy and human review: Appendix C and Listings 8-12, pp. 15-17: GPT-4o detector taxonomy, 99.31% no, 0.69% rewritten and five annotators
• Filtering bias and population coverage: Limitations, pp. 9-10: positivity bias and underrepresentation of rural, elderly, offline and under-resourced communities
• Theorem 1 failure: Theorem 1 and proof, pp. 19-20, equations 22/24 and A-D: unbiased weights treated as exact distribution equality, Θ(1) normalization and unsquared error-order problems
• Theorem 2 failure: Theorem 2 and proof, pp. 19-20, equations 23/25 and A-C: feasible couplings may contain zeros and the derived gap is epsilon log(N-dagger M), not uniform O(epsilon)
• Template residues: Running headers Trovato et al. throughout and final line p. 20: Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009
• Absence of reproducible artifacts: Complete PDF links plus targeted GitHub and Hugging Face searches inspected 15 Jul 2026: only generic model/training references; no paper-specific code, persona data or result repository found
• Comprehensive visual reading and checking: All 20 pages rendered and inspected, including Figures 1-4, Tables 1-10, equations 1-25, prompts, privacy examples, limitations, proofs and references; checked 15 Jul 2026