Synthetic Personality Review

Value-Spectrum: Quantifying Preferences of Vision-Language Models via Value Decomposition in Social Media Contexts

Applications, bias, and safety2025ACL AnthologyApproved editorial review

Authors: Jingxuan Li, Yuning Yang, Shengqi Yang, Linfan Zhang, Ying Nian Wu

Keywords: Computation and Language, ACL 2025

Source: Open primary source (opens in a new tab)

5
Authors
15
Findings
35
Limitations
18
Evidence

Editorial summary

English

Value-Spectrum studies vision-language model responses and simulated browsing behavior on social-media content, using Schwartz’s ten values as retrieval labels. The stated corpus contains 50,191 videos from TikTok (39%), Instagram (32%), and YouTube (29%), dated 31 July to 31 October 2024; each record stores a link, metadata, and one screenshot. For the static benchmark, the authors write ten keywords per value, retrieve five CLIP-nearest images per keyword, and ask eight VLMs whether they like each image. The percentage of yes answers over 50 images per value is called preference intensity. Gemini 2.0 Flash answers yes on 82–94% depending on the value, while InternVL2 ranges from 26–54%. Those differences also reflect general acquiescence, format following, model capability, and image selection, not values alone. There are no neutral items, counterbalanced images, repeat generations, inference parameters, uncertainty intervals, or statistical tests, so the design does not identify intrinsic preferences. In a second phase, five VLMs receive PersonaChat biographies and decide whether to remain for 45 seconds or skip each video. Ten 100-video runs per model and platform compare model-accepted content in the first and last 50 items. A Simple strategy uses yes/no; ISQ combines four 1–10 ratings and two binary answers, staying when its normalized composite exceeds 60%. Changes jointly depend on the model, prompt, served content, and each platform’s recommender. Under ISQ, for example, relative change on TikTok ranges from −2.0% for Qwen-VL-Plus to 51.9% for Gemini 1.5 Pro; YouTube reaches 34.9% for Claude, and Instagram 16.4% for GPT-4o. There is no same-sequence control, uncertainty estimate, or causal test of personality adoption. Two human checks provide narrower support: three ratings for each of 1,500 screenshots, 500 per platform, judge the two-second frame representative of its video in 90.4% of cases; in a separate stratified sample of 500 screenshot-video pairs with three annotators, 90.6% of screenshot judgments and 87.6% of video judgments say the content reflects its assigned value. These rates partially validate content retrieval, but do not measure inter-rater agreement or validate model preferences. Visual review also confirms an unresolved “Figure ??” reference. The cultural analysis applies GPT-4o captions and NER/keyword rules to 2,614 images; categories may overlap, their counts total 2,797 and percentages 107%, despite the table adding a 2,614/100% total row. The linked repository contradicts the abstract’s claim that complete code and data are available: commit 6f45f1d9cc071a034dfaecfe64bc75dfb30babb6 contains only a README, license, and images, while its TODO leaves the dataset, evaluation, agent, and annotation code pending. The defensible finding is that VLMs have markedly different yes rates under one specific protocol and that persona prompts can alter a signal fed to real recommenders; the study does not establish internal values, stable personality, or causal steerability.

Español

Value-Spectrum estudia respuestas y navegación simulada de modelos de visión y lenguaje ante contenido de redes sociales, usando los diez valores de Schwartz como etiquetas de recuperación. El corpus declarado contiene 50.191 vídeos de TikTok (39 %), Instagram (32 %) y YouTube (29 %), publicados entre el 31 de julio y el 31 de octubre de 2024; de cada vídeo se conserva un enlace, metadatos y una captura. Para el benchmark estático, los autores redactan diez palabras por valor, recuperan mediante CLIP cinco imágenes por palabra y preguntan a ocho VLM si les gusta cada imagen. La proporción de «sí» en 50 imágenes por valor se denomina intensidad de preferencia. Gemini 2.0 Flash responde afirmativamente entre 82 % y 94 % según el valor, mientras que InternVL2 queda entre 26 % y 54 %; estas diferencias reflejan también propensión general a asentir, capacidad de seguir el formato y selección de imágenes, no sólo valores. El diseño no incluye ítems neutrales, imágenes contrabalanceadas, repetición de generaciones, parámetros de inferencia, intervalos ni tests. Por ello no identifica preferencias «intrínsecas». En una segunda fase, cinco VLM reciben biografías de PersonaChat y deciden si permanecer 45 segundos o saltar cada vídeo. Se ejecutan diez recorridos de 100 vídeos por modelo y plataforma, comparando la tasa de contenido considerado relevante en los primeros y últimos 50. Una estrategia simple usa sí/no; ISQ suma cuatro ratings de 1–10 y dos respuestas binarias, con umbral superior al 60 % para permanecer. Los cambios dependen simultáneamente del modelo, el prompt, el contenido servido y el recomendador de cada plataforma. Por ejemplo, con ISQ el cambio relativo en TikTok va de −2,0 % para Qwen-VL-Plus a 51,9 % para Gemini 1.5 Pro; en YouTube llega a 34,9 % para Claude y en Instagram a 16,4 % para GPT-4o. No hay comparación sobre las mismas secuencias, estimación de incertidumbre ni prueba causal de adopción de personalidad. Dos validaciones humanas aportan evidencia más estrecha: tres anotaciones por 1.500 capturas, 500 por plataforma, consideran representativa del vídeo una captura tomada a los dos segundos en el 90,4 % de los casos; en otra muestra estratificada de 500 pares y tres anotadores, el 90,6 % de los juicios sobre capturas y el 87,6 % sobre vídeos responden que el contenido refleja el valor asignado. Esas tasas validan en parte la recuperación de contenido, pero no miden acuerdo entre anotadores ni validez de las respuestas de los modelos. La auditoría visual confirma además una referencia «Figure ??» sin resolver. El análisis cultural usa captions de GPT-4o y reglas NER/palabras clave sobre 2.614 imágenes; las categorías pueden solaparse, sus cuentas suman 2.797 y sus porcentajes 107 %, aunque la tabla añade un total 2.614/100 %. El repositorio enlazado contradice la frase del abstract que afirma disponer de código y datos completos: en el commit 6f45f1d9cc071a034dfaecfe64bc75dfb30babb6 sólo hay README, licencia e imágenes, y su TODO deja pendientes dataset, evaluación, agente y código de anotación. El resultado defendible es que diferentes VLM muestran tasas de «sí» muy distintas bajo un protocolo concreto y que un agente con prompts de persona puede alterar una señal que alimenta recomendadores reales; no demuestra valores internos, personalidad estable ni steerability causal.

Research question

Do different VLMs produce different patterns of acceptance toward images retrieved using Schwartz values, and can a persona prompt change, through stay or skip decisions, the content that TikTok, YouTube, and Instagram recommend?

Method

50,191 short videos and one screenshot at two seconds per video are collected. CLIP indexes the screenshots. Ten words chosen by the authors for each of ten values retrieve five images per word; eight VLMs answer yes/no and explain their response, and the yes rate across 50 images is presented as preference. For induction, five VLMs receive PersonaChat biographies and navigate ten runs of 100 videos per platform. They stay 45 seconds when a simple decision or an ISQ score exceeds the threshold; the relative change between the first and last 50 videos is calculated. There is human validation of screenshot representativeness and content-value alignment, a comparison of image versus textual description, and an automated cultural analysis.

Sample: The corpus contains 50,191 videos, but the preference benchmark uses 50 screenshots retrieved for each value and model. The persona experiment declares N=10 runs of 100 videos per model and platform, comparing blocks of 50. The human validations use 1,500 pairs for frame representativeness and 500 stratified pairs for alignment with values. The number of unique workers and their demographics are not reported.

Findings

  • Across eight VLMs, the mean yes rate per value ranges from globally low profiles, such as InternVL2 (26–54 %), to very affirmative responses from Gemini 2.0 Flash (82–94 %).
  • The average across models is 73.3 for Universalism and Benevolence, 72.8 for Tradition, and 55.0 for Stimulation.
  • GPT-4o scores 90 on Universalism, 88 on Benevolence, and 56 on Stimulation; the values are percentages of yes, not calibrated psychometric scales.
  • CogVLM2 reaches 90 on Power, whereas BLIP-2 only 28; a difference that may include responsiveness and acquiescence bias.
  • In the simple strategy, the relative change of accepted content varies by platform and model: GPT-4o registers 55.26 % on TikTok but −4.0 % on YouTube and −9.82 % on Instagram.
  • With ISQ, Gemini 1.5 Pro reaches 51.9 % change on TikTok, Claude 34.9 % on YouTube, and GPT-4o 16.4 % on Instagram.
  • Qwen-VL-Plus worsens with ISQ on TikTok (−2.0 %) and Instagram (−6.0 %), which contradicts a uniform improvement.
  • The introductory claim that Claude achieves the highest ISQ alignment does not clearly match the tables: Gemini has the highest mean change across platforms and CogVLM the highest absolute rates of accepted content.
  • 90.4 % of 4,500 judgments considers that the screenshot at two seconds represents the video; 8.8 % considers it non-representative, and the remaining 0.8 % is not broken down in the main text.
  • In label validation, 90.6 % of judgments on screenshots and 87.6 % on videos answer yes to alignment with the assigned value.
  • Label validation does not publish inter-annotator agreement or reliability; a high yes rate against a preselected label does not equate to agreement among judges.
  • The modality comparison uses 500 images and shows higher rates when captions from other VLMs are given to LLMs; it presents no tests despite rating differences as significant.
  • The cultural analysis attributes 78.6 % of signals to Western and 13.9 % to Japanese, but admits multiple categories; counts and percentages add up to 2,797 and 107 %, not the total 2,614/100 % shown.
  • Visual review of the 20 pages confirms tables and formulas, and detects an unresolved reference to Figure ?? in Appendix G.
  • As of 15 July 2026, the official repository contains no dataset or executable code and keeps those artifacts as TODO, despite the complete availability claimed by the abstract.

Limitations

  • The main variable is a yes/no liking response, not a validated measure of Schwartz values or personality.
  • Images are selected from words chosen to represent each value; the content, the word, and the value are confounded.
  • Only five images per word and ten words per value are used, with no sensitivity analysis to another selection of terms or screenshots.
  • A yes rate mixes preference, acquiescence, model safety, format following, refusal policy, and visual quality.
  • There is no neutral baseline, control of overall yes rate, counterbalanced negative/positive images, or comparison with human preferences.
  • Exact versions of most endpoints/checkpoints, temperature, top-p, seed, query date, retries, or response parser are not reported.
  • There are no replications of the static benchmark, intervals, standard errors, or hypothesis tests for any difference between values or models.
  • The text repeatedly uses "significant" descriptively without inferential statistics.
  • The deviation across ten values is interpreted as preference specificity, although it is not uncertainty and depends on the range of chosen images.
  • BLIP-2 produces short or inconclusive responses, so comparing it using the same parser with instructed models may measure capability rather than preference.
  • The study calls intrinsic the responses provoked by explicit prompts and a set of images constructed by the authors.
  • The models used in the static benchmark do not exactly match those in the persona experiment: Gemini 2.0/1.5, Qwen2.5/Qwen-VL, and CogVLM2/CogVLM.
  • It is not justified how PersonaChat personas are chosen, whether they repeat across models, or whether the initial sequences are comparable.
  • The first and last 50 videos are different contents served adaptively; the change does not isolate the effect of the prompt from the algorithm, order, or session drift.
  • The relative change can be amplified when the initial rate is low, such as the 55.26 % of GPT-4o on TikTok which corresponds to going from 7.6 % to 11.8 %.
  • No control without persona, permuted persona, random decision, fixed stay, or same video sequence is included.
  • Staying 45 seconds or skipping are different and sufficient interventions to train the recommender; they do not prove that the VLM has adopted a personality.
  • ISQ directly asks about alignment with the persona and desire to act, and sums those yes with weight ten; it improves the signal by construction and does not validate an independent construct.
  • The ISQ threshold is presented as an example, and no calibration, sensitivity analysis, or validation of its weights is provided.
  • The ten trials are not accompanied by dispersion, intervals, individual results, or tests; only aggregate percentages are published.
  • Prior history, geography, account, cookies, time, recommender version, or initial personalization of the platforms are not controlled.
  • The conclusion that TikTok is optimal is not causally supported without comparable conditions or temporal replication.
  • Frame validation evaluates general representativeness, not whether the frame preserves the specific value or the decisive temporal information.
  • The alignment sample is stratified by labels already assigned by the pipeline, and the three annotators know Schwartz; there is no open blind labeling.
  • Aggregate percentages of judgments are reported, but no per-item consensus, inter-annotator agreement, kappa, disagreement across values, or per-platform data.
  • The demographics of annotators, the number of unique MTurk workers, or inclusion criteria beyond iterative pilots are not reported.
  • The cultural table combines overlapping categories with a Total 100 % row, without formally explaining multiple denominators; its percentages add up to 107 %.
  • The cultural analysis depends on GPT-4o captions and entity/word rules, without human validation, error metrics, or an exhaustive list of rules.
  • Cultural predominance cannot be attributed to the platforms in general because the collection is a walk conditioned by undocumented accounts and recommenders.
  • Automated capture of content and social network links raises privacy, copyright, consent, and terms of service issues that the ethics section does not analyze.
  • The treatment of faces, minors, account names, deleted content, retention, access, anonymization, or deletion requests is not explained.
  • No ethical/IRB approval, scraping risk assessment, or platform permission is documented.
  • The official repository, at its inspected commit, contains only README, license, and images; it does not allow reproduction of collection, prompts, parsing, tables, or analysis.
  • The dataset is not available despite the PDF claiming otherwise, so duplicates, dates, labels, sensitive content, or broken links cannot be audited.
  • The document retains a "Figure ??" reference and presents introductory claims that do not unambiguously fit its ISQ tables.

What the study does not establish

  • It does not demonstrate that VLMs possess intrinsic human values or internal preferences.
  • It does not validate the yes rate as a psychometric scale of Schwartz values.
  • It does not separate preferences from acquiescence bias, visual capability, or instruction following.
  • It does not demonstrate stable personality, identity, consciousness, or human traits in the models.
  • It does not demonstrate that an induced persona persists outside the evaluated session, platform, or prompt.
  • It does not causally identify the prompt as the origin of recommender changes.
  • It does not establish that ISQ measures personality better; it explicitly incorporates questions about alignment and action.
  • It does not demonstrate that TikTok is universally better for evaluating role-playing.
  • It does not establish statistically significant differences between models, values, modalities, or platforms.
  • It does not demonstrate balanced cultural diversity of the corpus.
  • It does not demonstrate that a single screenshot adequately represents all videos or all their values.
  • It does not provide inter-annotator agreement or psychometric validation of the human labels.
  • It does not allow reproduction of the results with the currently published artifacts.
  • It does not certify that the complete code and data are available in the linked repository.
  • It does not evaluate impact on users, persuasion, well-being, discrimination, safety, or recommender manipulation.

Traceability

Scope: Full text

Version: ACL 2025 Main Conference long paper, pages 9591–9610; DOI 10.18653/v1/2025.acl-long.472

Consulted source: https://aclanthology.org/2025.acl-long.472.pdf

Review: Codex full-text, bilingual-fidelity, visual, construct-validity, acquiescence, retrieval-confounding, sampling, model-version, adaptive-recommender, causal-inference, repeated-measures, inferential-statistics, formula, human-annotation, cultural-analysis, artifact-availability, privacy, ethics and reproducibility audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o; exact snapshot and inference configuration not reported
  • Gemini 2.0 Flash; exact snapshot and inference configuration not reported
  • Claude 3.5 Sonnet; exact snapshot and inference configuration not reported
  • DeepSeek-VL2; exact checkpoint and inference configuration not reported
  • Qwen2.5-VL-Plus; exact endpoint/checkpoint and inference configuration not reported
  • InternVL2; exact checkpoint and inference configuration not reported
  • CogVLM2; exact checkpoint and inference configuration not reported
  • BLIP-2 2.7B; exact checkpoint and inference configuration not reported
  • Persona experiment: GPT-4o, Gemini 1.5 Pro, Qwen-VL-Plus, Claude 3.5 Sonnet, and CogVLM

Instruments and metrics

  • Schwartz ten basic human values used as retrieval categories
  • Ten author-selected keywords per value
  • CLIP vector retrieval with five images per keyword
  • One-word yes/no liking prompt plus explanation and brief description
  • Percentage of yes responses over 50 images per value
  • PersonaChat demographic persona prompts
  • Simple stay/skip strategy
  • Inductive Scoring Questionnaire composite with a 60% threshold
  • Relative change between accepted content in the first and last 50 videos
  • MTurk screenshot representativeness judgments
  • Three-annotator screenshot/video value-alignment judgments
  • GPT-4o captioning plus NER and keyword matching for cultural signals

Data used

  • Value-Spectrum: 50,191 unique short-video links, screenshots, platform and post-date metadata collected from TikTok, Instagram Reels, and YouTube Shorts
  • Static preference subset: 500 retrieved screenshots per evaluated VLM, 50 per Schwartz value
  • Persona navigation runs: ten 100-video trials per model and platform for each reported strategy
  • Single-frame validation: 1,500 screenshot-video pairs, three ratings per pair, 4,500 ratings
  • Value-label validation: 500 screenshot-video pairs, 50 per value, three annotators, 1,500 screenshot and 1,500 video judgments
  • Input-modality ablation: 500 images, 50 per value
  • Cultural-signal sample: 2,614 images

Evidence and location

  • Identity, authors, publication, pages, DOI, and complete abstract: ACL final PDF p. 1 (proceedings p. 9591) and ACL Anthology record 2025.acl-long.472
  • Declared contributions and claim of intrinsic preferences: PDF pp. 1–2 (9591–9592), Abstract and Introduction
  • Corpus of 50,191 videos, platforms, dates, one screenshot, and CLIP retrieval: PDF pp. 3–4 (9593–9594), section 3 and Figures 3–4
  • Ten words per value, five images per word, and prompts: PDF pp. 4–5 (9594–9595), section 4.1
  • Yes rates across eight models: PDF pp. 5–6 (9595–9596), Figures 5–7 and Table 1
  • Change of families/models between experiments: PDF p. 6 (9596), section 5.1
  • PersonaChat, stay/skip, and change formula: PDF pp. 6–7 (9596–9597), Simple Strategy and Equation I_avg
  • Ten runs, one hundred videos, and blocks of fifty: PDF p. 7 (9597), paragraph following I_avg
  • ISQ questions, weights, and threshold: PDF p. 7 (9597), Inductive Scoring Questionnaire Strategy and Equation S%
  • Results by platform and strategy: PDF pp. 7, 16 (9597, 9606), Figure 8 and Tables 3–8
  • Comparison of visual versus image descriptions: PDF pp. 8, 13, 20 (9598, 9603, 9610), Discussion, Appendix E and Table 10
  • Screenshot representativeness validation: PDF pp. 8, 13, 20 (9598, 9603, 9610), Single Frame Screenshot Representation, Appendix D and Figure 15
  • Declared limitations and ethical considerations: PDF p. 9 (9599), sections 8–9
  • Cultural analysis and non-partitional sum: PDF pp. 13–14 (9603–9604), Appendix F and Table 2
  • Human label validation and absence of inter-annotator agreement: PDF pp. 14–17 (9604–9607), Appendix G and Table 9
  • Unresolved editorial reference: PDF p. 15 (9605), Appendix G.1: Figure ??
  • Visual inspection: All 20 pages of the final ACL PDF rendered and visually inspected, including formulas, ten tables and fifteen figures; checked 15 Jul 2026
  • Absence of dataset and code despite the abstract's claim: Official GitHub repository Jeremyyny/Value-Spectrum at commit 6f45f1d9cc071a034dfaecfe64bc75dfb30babb6: README TODO leaves dataset, evaluation, VLM agent and annotation code pending; repository tree contains README, MIT license and image assets only; checked 15 Jul 2026