German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies

Society, culture, and collective behavior2025arXivApproved editorial review

Authors: Jens Rupprecht, Leon Fröhling, Claudia Wagner, Markus Strohmaier

Keywords: Survey-grounded personas, ALLBUScompact 2023, Population response simulation, Jensen-Shannon distance, Representativeness and data quality

Source: Open primary source (opens in a new tab)

4
Authors
9
Findings
19
Limitations
4
Evidence

Editorial summary

English

The paper introduces German General Social Survey Personas (GGSS Personas), the version-2 name for the collection previously called German General Personas. It converts records from 5,246 ALLBUScompact 2023 participants into prompts. Each persona contains a fixed sociodemographic block and a TOP-k block. To construct the latter, the authors remove paradata, core sociodemographics, and 27 selected outcome variables; they then train 406 random forests, each using the other 405 variables, and aggregate each model's ten largest feature importances. This produces a global ranking of 380 attributes and variants with k from 2 to 380. Questionnaire splits make the personas incomplete: TOP-2 contains 1.34 attributes on average and TOP-380 contains 242.27; from TOP-16 onward, no persona contains every possible attribute.

The evaluation asks five open-weight models, Mistral-7B-Instruct-v0.3, Llama-3.1-8B-Instruct, Qwen3-8B, Gemma-3-12B-it, and Llama-3.3-70B-Instruct, to answer 27 questions, three from each of nine topics. The target variable is not included literally in the prompt. Individual answers are aggregated and compared with the observed distribution using Jensen-Shannon distance (JSD). Baselines are random forests using the same TOP-k feature sets and 2 to 2,048 training examples; 20% of participants are held out for testing. Seeds, repeated runs, intervals, and significance tests are not reported. Llama-3.3-70B with TOP-2 is the best observed configuration. It beats the best tested random forest on 13 of 27 questions and on the mean of five of nine topics when the baseline uses up to 512 cases. Adding attributes is not monotonically beneficial and often worsens JSD.

The representativeness result is weaker than the resource narrative. When a representative 500-person ALLBUS subset is compared with income-, conservative-, or student-oversampled subsets, PersonaHub, and no persona, distributions cluster closely; the paper itself concludes that representativeness has little influence in this test. Responses are also aggregated without reported survey weights even though the official archive describes a disproportional stratified multistage probability design. The empirical target is therefore the raw 2023 ALLBUS respondent distribution, not demonstrated fidelity to the entire German population. Marginal JSD does not measure individual correctness, subgroup fidelity, multivariate dependence, or real behavior. The manuscript is ambiguous about whether LLMs are compared on all 5,246 personas while random forests are evaluated only on the 20% test set, and it does not explain how the 1.44-9.91% invalid responses enter the metric.

The collection provides key-value and Gemini-generated prose formats. Two coauthors inspect 80 personas from TOP-2 through TOP-16 and judge 69 faithful: 11, or 13.75%, contain omissions or misrepresentations. No inter-rater agreement is reported, and TOP-32 and larger variants are not systematically validated. The appendix acknowledges severe TOP-380 loops. An audit of the public GitHub ZIP at the 22 January 2026 commit, which may differ from the later GESIS archive, quantifies the problem: rows above 100,000 characters occur in 5 TOP-64, 11 TOP-128, 79 TOP-256, and 556 TOP-380 personas. TOP-380 has 690 rows above 10,000 characters and reaches 446,981; TOP-16 already contains a 66,349-character row dominated by a repeated symbol. Every JSONL parses and has the advertised row count, but container integrity does not establish semantic fidelity.

GESIS preserves the dataset as ZA9089 v1.0.0 with a DOI and codebook. The public repository nevertheless lacks construction and evaluation code, model revisions, seeds, raw generations, invalid-output decisions, and figure data. Its README retains the old name, gives a placeholder clone URL, leaves citation and contact as TBA, and refers to a missing LICENSE file. The paper names Gemini-2.5-flash as the prose generator, while current GESIS metadata says Gemini-2.5-flash-lite. ALLBUScompact coarsens sensitive fields and removes fine geography, but the official edition retains a linkable respid alongside combinations of household, income, migration, religious, and political attributes; residual linkage and attribute-inference risk remains, although this audit does not claim a reidentification. The solid contribution is a versioned corpus and a preliminary aggregate-alignment benchmark. It does not establish faithful digital twins, survey replacement, universal representativeness, or complete reproducibility.

Español

El artículo presenta German General Social Survey Personas (GGSS Personas), nombre adoptado en la versión 2 para la colección antes llamada German General Personas. Convierte en prompts los registros de 5.246 participantes de ALLBUScompact 2023. Cada persona contiene un bloque sociodemográfico fijo y un bloque TOP-k. Para construir este último, los autores eliminan paradata, campos sociodemográficos y 27 variables elegidas como resultados; después entrenan 406 random forests, cada uno con las otras 405 variables, y agregan las diez importancias principales de cada modelo. De ahí sale un ranking global de 380 atributos y versiones con k entre 2 y 380. Como el cuestionario se reparte en módulos, las personas están incompletas: TOP-2 contiene de media 1,34 atributos y TOP-380, 242,27; a partir de TOP-16 ninguna persona tiene todos los atributos posibles.

La evaluación pide a cinco modelos abiertos, Mistral-7B-Instruct-v0.3, Llama-3.1-8B-Instruct, Qwen3-8B, Gemma-3-12B-it y Llama-3.3-70B-Instruct, que respondan 27 preguntas, tres en cada uno de nueve temas. La variable objetivo no aparece literalmente en el prompt. Las respuestas individuales se agregan y se comparan con la distribución observada mediante distancia de Jensen-Shannon (JSD). Los baselines son random forests con los mismos conjuntos TOP-k y entre 2 y 2.048 ejemplos de entrenamiento; el 20% de los participantes se reserva como test. No se publican semillas, repeticiones, intervalos ni pruebas de significación. Llama-3.3-70B con TOP-2 es la mejor configuración observada. Supera al mejor random forest probado en 13 de 27 preguntas y en la media de cinco de nueve temas cuando el baseline usa hasta 512 casos. Añadir atributos no mejora de forma monótona y a menudo empeora la JSD.

El resultado sobre representatividad es más débil que la narrativa del recurso: al comparar una submuestra ALLBUS de 500 personas con submuestras sobrerrepresentadas por ingresos, conservadurismo o estudiantes, PersonaHub y ausencia de persona, las distribuciones quedan próximas y el propio artículo concluye que la representatividad tiene poca influencia en esa prueba. Además, se agregan respuestas sin informar uso de pesos, aunque el archivo oficial describe un muestreo probabilístico multietápico, estratificado y desproporcional. Por tanto, el objetivo empírico es la distribución bruta de participantes ALLBUS 2023, no una estimación demostrada de toda la población alemana. La JSD marginal tampoco mide acierto individual, fidelidad por subgrupos, dependencias multivariantes ni comportamiento real. El texto es ambiguo sobre si el LLM se compara sobre las 5.246 personas mientras el random forest se evalúa solo en el 20% de test, y no explica cómo trata el 1,44-9,91% de respuestas inválidas.

La colección ofrece formato clave-valor y prosa generada por Gemini. Dos coautores revisan 80 personas de TOP-2 a TOP-16 y consideran fieles 69: 11, o 13,75%, contienen omisiones o representaciones erróneas. No se informa acuerdo entre anotadores y no se valida sistemáticamente TOP-32 o superiores. El apéndice reconoce bucles graves en TOP-380. La auditoría del ZIP público de GitHub, commit de 22 de enero de 2026 y por tanto no necesariamente idéntico al archivo GESIS posterior, cuantifica el problema: hay filas de más de 100.000 caracteres en 5 personas TOP-64, 11 TOP-128, 79 TOP-256 y 556 TOP-380; esta última versión tiene 690 filas por encima de 10.000 caracteres y alcanza 446.981. También existe una fila TOP-16 de 66.349 caracteres dominada por un símbolo repetido. Los JSONL parsean y tienen el número esperado de filas, pero eso no garantiza fidelidad semántica.

GESIS preserva el dataset como ZA9089 v1.0.0 con DOI y codebook. Sin embargo, el repositorio público no incluye el código de construcción o evaluación, revisiones de modelos, semillas, generaciones, decisiones sobre inválidos ni datos de figuras. Su README aún usa el nombre antiguo, ofrece una URL de clonación de plantilla, deja cita y contacto como TBA y menciona un LICENSE inexistente. El paper dice Gemini-2.5-flash y la ficha GESIS, Gemini-2.5-flash-lite. ALLBUScompact reduce detalle y elimina geografía fina, pero la edición oficial conserva un respid enlazable y combinaciones sensibles de hogar, ingresos, migración, religión y política; esto deja riesgo residual de enlace e inferencia, sin que esta auditoría afirme una reidentificación. La contribución sólida es un corpus versionado y un benchmark preliminar de alineamiento agregado. No demuestra gemelos digitales fieles, sustitución de encuestas, representatividad universal ni reproducibilidad completa.

Research question

Can a collection of persona prompts derived from ALLBUS approximate aggregate response distributions with LLMs without specific training, and how do the number of attributes and the composition of the sample change that approximation?

Method

5,246 records from ALLBUScompact 2023 are transformed into key-value and textual personas. A global ranking, obtained by aggregating importances from 406 random forests, defines TOP-k variants of 2 to 380 attributes. Five LLMs answer 27 variables excluded from the prompt; their marginal distributions are compared using JSD with ALLBUS and with random forests trained with n=2-2,048. Subsamples of 500 representative, overrepresented, PersonaHub, and no-persona individuals are also tested. Two coauthors review 80 short textual personas.

Sample: The collection contains 5,246 personas per variant and 20 variants archived in GESIS. 27 outcomes are evaluated, three for each of nine topics. The baseline reserves a fixed 20% for test and uses from 2 to 2,048 remaining participants for training. Composition comparisons use 500 personas per subset. Textual validation covers 80 cases: 20 from TOP-2, TOP-4, TOP-8, and TOP-16. No seeds, repetitions, or effective N after invalid responses are reported.

Findings

  • Llama-3.3-70B with TOP-2 is the configuration with the lowest observed JSD in the evaluated set.
  • That configuration outperforms the best tested random forest in 13 of 27 outcomes and in five of nine thematic means with up to 512 training cases.
  • The LLM advantage is greater against baselines with very little data and decreases as the training sample grows.
  • Adding attributes does not improve monotonically; TOP-2 outperforms more extensive variants in the tested configurations.
  • Manipulating representativeness produces small and highly topic-dependent differences.
  • Invalid response rates range from 1.44% to 9.91% and increase in some alternative populations.
  • Manual validation considers 69 of 80 short textual personas faithful; 13.75% present some error.
  • The audited GitHub artifact contains hundreds of runaway textual generations in TOP-256 and TOP-380 and isolated failures from TOP-16 onward.
  • The corpus is archived with a DOI, but there is not enough public code to reproduce its construction or its experiments.

Limitations

  • The sample is aggregated without published weights despite a disproportionate multi-stage stratified design.
  • Representing ALLBUS 2023 participants does not automatically equate to representing the entire German population.
  • Only marginal distributions are evaluated; individual accuracy, subgroups, joint dependencies, and external behavior are missing.
  • It is unclear whether the LLM and the random forest are evaluated against exactly the same target participants.
  • The best configuration is chosen and the main result is reported on the same 27 tasks, without a task holdout.
  • Only a random set of 27 variables is taken and neither its seed nor repetition is published.
  • There are no intervals, contrasts, multiplicity correction, decoding repetitions, or repeated splits.
  • The treatment of invalid responses in the JSD is not specified.
  • The impurity-importance ranking does not control for cardinality, correlation, multicollinearity, or missingness.
  • Attribute selection uses the entire dataset, including participants later reserved for test.
  • The knowledge cutoffs do not rule out contamination, especially from repeated questions from previous waves.
  • Human validation is internal, small, and excludes all textual variants from TOP-32 to TOP-380.
  • Inter-annotator reliability is not reported and per-row labels are not released.
  • The public textual artifact has loops and massive corruptions that the paper does not quantify.
  • Code, environments, model revisions, seeds, outputs, decisions on invalid responses, and figure data are missing.
  • The paper and GESIS disagree between Gemini-2.5-flash and Gemini-2.5-flash-lite.
  • The record and the README retain old nomenclature and the repository lacks the LICENSE it announces.
  • The richness of attributes and the linkable respid leave a residual privacy risk not analyzed in the paper.
  • The work remains an arXiv v2 preprint with no established peer-reviewed venue.

What the study does not establish

  • That the personas are faithful digital twins of real individuals.
  • That the LLM correctly predicts each participant's response.
  • That a low marginal JSD implies subgroup, joint, causal, or behavioral fidelity.
  • That the unweighted corpus reproduces the German population for any task.
  • That the representativeness of the collection generally improves the LLM's responses.
  • That TOP-2 is optimal outside the tasks, models, and prompts used to select it.
  • That LLMs replace surveys, participants, or human ground truth.
  • That the comparison with the random forest is free of advantages from selection or target population.
  • That the complete textual collection is semantically faithful or usable without filtering.
  • That the declared cutoffs eliminate all possibility of contamination from ALLBUS.
  • That the dataset is free of linkage risk or inference of sensitive attributes.
  • That the pipeline and the figures can be reproduced with the published artifacts.
  • That the dataset DOI or arXiv v2 implies peer review.

Traceability

Scope: Full text

Version: arXiv:2511.21722v2, 20 pages; GESIS ZA9089 v1.0.0 and public GitHub data snapshot also audited

Consulted source: https://arxiv.org/abs/2511.21722

Review: Codex 20-page arXiv-v2 visual, GESIS dataset/codebook, survey-design/metric, full-text artifact, privacy, reproducibility and claim-boundary audit, 2026-07-16

Approval: Codex fidelity pass, 2026-07-16

English translation: approved, 2026-07-18

Models evaluated

  • Mistral-7B-Instruct-v0.3
  • Llama-3.1-8B-Instruct
  • Qwen3-8B
  • Gemma-3-12B-it
  • Llama-3.3-70B-Instruct (8-bit)
  • Gemini-2.5-flash or Gemini-2.5-flash-lite prose generator
  • scikit-learn RandomForestClassifier baselines

Instruments and metrics

  • ALLBUScompact 2023 question and response labels
  • Core sociodemographic plus TOP-k persona construction
  • Global random-forest feature-importance ranking
  • JSON-like key-value persona format
  • LLM-generated full-text persona format
  • Twenty-seven held-out survey outcome variables across nine topics
  • Normalized Jensen-Shannon distance
  • Representative and oversampled 500-person population comparison
  • Manual full-text fidelity annotation on 80 personas
  • Invalid-response rate

Data used

  • German General Social Survey ALLBUScompact 2023 (ZA8831)
  • German General Social Survey Personas, GESIS ZA9089 v1.0.0
  • Public German-General-Personas GitHub artifact snapshot
  • PersonaHub comparison personas
  • Income-, ideology- and student-oversampled ALLBUS subsets

Evidence and location

  • Design, construction, models, results, annexes, limitations, and ethics: arXiv:2511.21722v2, 20 PDF pages; all pages rendered and visually inspected
  • Archived version, sample, sampling, formats, and linkage via respid: GESIS ZA9089 v1.0.0 landing page and 10-page codebook, DOI 10.4232/1.14707
  • JSONL integrity, textual quality, files, and public reproducibility: germanpersonas/German-General-Personas commit ed0e1a08e6bf19f5b27b181e7d53655170c3ac04
  • Representativeness, metric, privacy, artifact quality, and limits of claims: reports/verification/article-269-ggss-personas-representativeness-metric-fulltext-artifact-and-reproducibility-audit.json