Persona-Based Simulation of Human Opinion at Population Scale

Research question

Whether a semi-structured person inferred from public social media traces recovers survey responses better and with more heterogeneity than seven demographics, and whether weighted banks of such persons can approximate public opinion on stable topics and recent events.

Method

A Painter concatenates chronologically all retrievable posts and generates via LLM a fixed-schema JSON and a narration; a Reasoner selects one option, confidence, and justification per question. The internal evaluation retains 52 non-demographic questions from Ipsos and compares mean accuracy per user against a demographic person, in addition to a position-weighted composite. The external evaluation uses Gemma-3-27B-IT, demographics, and raking; it answers abortion and immigration directly and uses Tavily searches for Epstein and Venezuela before comparing Twitter and Reddit aggregates with Pew, YouGov, and CBS.

Sample: Handles were received from 1,410 Twitter/X users and 893 from Reddit, with 452 persons providing both. After requiring a valid account, public content, and API retrievability, Table A1 contains 1,031 Twitter profiles and 774 Reddit profiles. Raking uses n=1,517 unique persons; by arithmetic, this implies 288 participants retained with both services, but the text does not explicitly reconcile that dependency. Approximately 61% are men and more than half hold a university degree; Reddit is 58.8% liberal and 76.1% under 45 years old. The normalized weights have SD 1.32, median 0.63, minimum 0.012, and maximum 14.42.

Findings

• Figure 1B reports demographics/SPIRIT of 49.5/48.9% on Gemma-3-4B, 52.9/59.6% on Llama-3.1-8B, 55.5/62.1% on Gemma-3-12B, 54.5/63.4% on Gemma-3-27B, 55.1/64.3% on GPT-5-mini, and 56.9/65.7% on GPT-5.2.
• SPIRIT improves five of six models, not all: Gemma-3-4B is 0.6 points worse. The 8-9 point improvement only describes the larger models.
• Gemma-3-4B and Llama-3.1-8B fail on a non-trivial fraction even after ten JSON retries, but no counts, rates, or a common sample per model are published.
• Across the 81 questions with GPT-5-mini, accuracy ranges from 0.245 on finance and 0.257 on technology to 0.832 on vote and 0.929 on military affairs.
• The standard deviations of Table F8 match exactly, after rounding, sqrt(p(1-p)); they describe the binary row correction, not variation across questions, users, or runs.
• The sequences (1,2) and (3,1) both produce a position-weighted composite of 5/3, refuting the claim that the same value only appears for identical sequences.
• In Venezuela Q14, agents/CBS are 20.1/13% major threat, 79.9/48% minor threat, and 0/39% not a threat; in Q19 they are 39.6/37% decrease, 11.8/56% no change, and 48.6/7% increase.
• The official YouGov and CBS pages publicly contain the question and the percentages used as benchmark, so a web search with that topic could retrieve the target.

Limitations

• The task measures prediction of responses, not formation or change of opinion, intervention, behavior, or longitudinal consistency.
• The baseline receives seven demographics and SPIRIT all retrievable posts; text-only, retrieval-only, and ablations of JSON, narration, features, and opinions are missing.
• Big Five, primal beliefs, identities, mental health, experiences, and other attributes are not contrasted with real measures; the Painter's confidence is not calibrated.
• No date ranges or a cutoff of posts prior to the Ipsos survey are reported, so the held-out does not guarantee temporal isolation.
• The position-weighted composite has collisions, mixes binary, ordinal, and nominal codes from different scales, and gives arbitrary extra weight to later questions.
• The off-by-one rate uses numerical distance on nominal variables and, on binary questions, counts any error as close; the approximately 83% is not a valid general measure of proximity.
• There is no majority/chance baseline per item, intervals, paired test, bootstrap by user, or variability across runs for the accuracy differences.
• Schema failures of small models have no denominators and may change the composition of users across models.
• Probabilistic recruitment from KnowledgePanel does not preserve representativeness in the subset that uses social media, posts, consents, provides a valid handle, and has retrievable text.
• Raking starts with equal weights, not design weights, and does not correct by itself non-ignorable selection; calibrating on 2024 vote may mechanically approximate political opinions.
• Extreme weights imply, as an approximate diagnostic, a design effect of 2.74 and an effective n close to 553, with no trimming, sensitivity, or survey variance published.
• It is not explained how the at least 288 users retained on both platforms are treated, nor whether each bank again reaches the margins separately.
• The external validation mixes organizations, dates, samples, and modes; abortion has one item and Epstein two, and there is no MAE, correlation, calibration, preregistered criterion, or baseline.
• Synthetic aggregates lack uncertainty from sampling, raking, overlap, generation, search, or runs, and the figure does not incorporate sizes or margins of error of the benchmarks.
• Tavily could retrieve the target percentages directly; without queries, URLs, snippets, summaries, and timestamps, benchmark contamination cannot be ruled out.
• Seeds, temperature, top-p, search parameters, number of runs, exact GPT snapshots, per-user truncation/context, GPU-hours, and cost are missing.
• There is no public code, outputs, derived persons, weights, processed data, logs, or scripts; release is promised only after acceptance.
• Reference 24 studies homogenization and low replication of GPT-3.5 as a participant, not inference of traits from social text, and does not support the cited phrase.
• Inferring religion, politics, mental health, identity, and experiences from posts poses privacy, profiling, and manipulation risks; per-attribute validation, fairness, re-identification, deletion/unlearning, and detailed governance are missing.

What the study does not establish

• It does not demonstrate that SPIRIT simulates how a person forms, changes, or applies opinions; it demonstrates prediction of retained responses.
• It does not attribute the improvement to a psychological representation versus having much more individual text.
• It does not validate that inferred traits, beliefs, identities, or experiences are correct, nor that confidence is calibrated.
• It does not demonstrate an advantage across all models nor a statistically reliable difference.
• It does not demonstrate human heterogeneity via the composite, because it is not unique and has no interpretable scale.
• It does not establish that 83% of responses are substantively correct or close in nominal and binary domains.
• It does not convert the linked social subset into a representative sample of all U.S. adults through raking.
• It does not provide an independent external validation as long as searches can observe the benchmark percentages.
• It does not justify calling the bank a credible source of public opinion with few items, large errors, no uncertainty, and no formal metric.
• It does not allow reproducing the results with public artifacts available at the review date.

Traceability

Scope: Full text

Version: arXiv:2603.27056v1

Consulted source: https://arxiv.org/abs/2603.27056v1

Review: Codex 52-page visual full-text, official arXiv metadata, official poll-source, GitHub artifact, construct, population-inference, statistical, search-leakage, ethics and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

• SPIRIT Painter/Reasoner
• Gemma-3-4B
• Llama-3.1-8B
• Gemma-3-12B
• Gemma-3-27B
• google/gemma-3-27b-it
• GPT-5-mini
• GPT-5.2

Instruments and metrics

• 81 preguntas de Ipsos KnowledgePanel, 52 no demográficas en el análisis principal
• Exact-match medio por usuario
• Tasa off-by-one
• Compuesto de respuestas ponderado por posición
• Esquema SPIRIT con Big Five, 26 primal world beliefs, valores, identidades, experiencias y opiniones
• Raking a seis márgenes
• Comparación exploratoria con encuestas Pew, YouGov y CBS
• Búsqueda web Tavily para eventos recientes

Data used

• Ipsos KnowledgePanel survey responses linked under consent to public Twitter/X and Reddit histories
• Pew Research Center Public Opinion on Abortion
• Pew Research Center 2024 immigration policy survey
• November 15-17 2025 Economist/YouGov Epstein poll
• November 19-21 2025 CBS News/YouGov Venezuela poll

Evidence and location

• SPIRIT design, internal evaluation, and per-model values: arXiv v1, pp. 1-7, Figure 1; all 52 PDF pages visually inspected
• Sample, linkage, posts, raking, and weighted composite: arXiv v1, pp. 13-17 and p. 20 Table A1
• Computation, retries, and schema failures: arXiv v1, pp. 19-21, Appendix B
• Dictionary, categories, off-by-one, and Venezuela deviations: arXiv v1, pp. 21-35, Tables C3, E4-E7 and F8-F10
• Complete prompts and inferred sensitive fields: arXiv v1, pp. 37-49, Appendix G
• Metadata, version, and absence of public repo: Official arXiv Atom/HTML and GitHub repository search inspected 2026-07-17
• Visible benchmark and risk of search contamination: Official Pew, YouGov and CBS pages cited by the manuscript, retrieved 2026-07-17
• Reference 24 does not support inference of social traits: Official arXiv metadata and abstract for arXiv:2302.07267v6 inspected 2026-07-17
• Comprehensive audit of construct, statistics, population, leakage, artifacts, and ethics: reports/verification/article-381-spirit-population-inference-composite-leakage-statistics-artifacts-and-ethics-audit.json