LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion

Evaluation and psychometric validity2026arXivApproved editorial review

Authors: Tianyi Zhang, Wei Shan, Yuan Zong, Tianhua Qi, Wenming Zheng

Keywords: Apparent personality recognition, Asynchronous video interviews, AVI-6, HEXACO, Facial action units, AU-text semantic fusion, Multimodal assessment, Observer-rated personality, Algorithmic hiring, Measurement validity

Source: Open primary source (opens in a new tab)

5
Authors
7
Findings
14
Limitations
6
Evidence

Editorial summary

English

This preprint studies automated apparent-personality prediction in simulated asynchronous video interviews. It does not induce a personality in an LLM; it attempts to reproduce scores assigned to candidates by human observers. The pipeline uses OpenFace to extract intensities for 17 facial action units, finds keyframes through whole-frame pixel changes, and selects a trait-specific AU subset through simulated annealing guided by an LSTM's MSE. GPT-4o converts seven-frame AU windows into facial-movement descriptions and iteratively summarizes them. Meta-Llama-3.1-8B-Instruct then fuses that text with an automatic transcript; LoRA and a linear head are trained separately for Honesty-Humility, Extraversion, Agreeableness, and Conscientiousness. The study uses AVI-6 and reports 644 participants split by person into 450 training, 64 validation, and 130 test cases. The official dataset paper clarifies the labels: 12 raters with nine hours of training applied BARS measures, with eight independent ratings per participant and four items per trait; inter-rater reliability ranges from ICC(1,8)=0.61 to 0.83. These are aggregated observer judgments of behavior shown in an interview response, not HEXACO self-reports or independent measurements of stable traits. The method obtains MSEs of 0.1555, 0.1380, 0.1597, and 0.1077, averaging 0.1402; mean MAE is 0.2982 and Pearson correlations range from 0.4654 to 0.7258. Against the paper's Longformer text-only baseline, MSE improves by 14.33% to 48.83%. The most informative ablation is more mixed: adding facial descriptions to the same architecture improves H, X, and C but worsens Agreeableness relative to the no-AU variant, from MSE 0.1390 to 0.1597. It is not the strongest reported AVI-6 result either. The challenge publication reports average MSEs of 0.12284 and 0.13724 for its top two systems, although this is not a fully controlled comparison because the documents disagree without explanation between 644/646 participants and 450/452 training cases. The defensible contribution is architectural: on this split and with these generated features, fusion helps reproduce observer impressions for three of four traits relative to the direct ablation. It does not show that AUs reveal true personality or validate post-hoc labels such as sincerity, empathy, or control. The OpenFace-to-GPT-4o-to-summary-to-Llama chain creates several opportunities for information loss or invention; no human semantic-fidelity study is reported, and the authors observe hallucinated temporal dynamics with single-frame windows. Tables provide one estimate without intervals, seed distributions, or a naive baseline. Seed counts and values are missing, while GPT-4o uses service defaults without a seed. The simulated-annealing pseudocode also fails to update the state when a worse solution is accepted. No code, derived data, predictions, adapters, or reproducible environment is released. Most importantly, despite the hiring frame, the paper reports no error analysis by gender, age, race, skin tone, camera quality, or intersections, and no biometric-privacy, consent, adverse-impact, job-performance-validity, or real-deployment study. It should be read as a benchmark experiment on apparent personality, not evidence for evaluating or hiring candidates.

Español

Este preprint estudia la predicción automatizada de personalidad aparente en entrevistas de vídeo asíncronas simuladas. No induce una personalidad en un LLM: intenta reproducir puntuaciones asignadas por observadores humanos a candidatos. El sistema extrae con OpenFace intensidades de 17 unidades de acción facial, localiza fotogramas clave mediante cambios globales de píxeles y selecciona por rasgo un subconjunto de AUs con simulated annealing guiado por el MSE de un LSTM. GPT-4o transforma ventanas de siete fotogramas en descripciones de movimientos faciales y resume iterativamente esas descripciones. Después, Meta-Llama-3.1-8B-Instruct fusiona ese texto con la transcripción automática de la respuesta; LoRA y una capa lineal se entrenan para predecir por separado Honesty-Humility, Extraversion, Agreeableness y Conscientiousness. El estudio usa AVI-6 y declara 644 participantes repartidos por persona en 450 entrenamiento, 64 validación y 130 test. La fuente oficial del corpus aclara qué significan las etiquetas: 12 evaluadores con nueve horas de formación aplicaron escalas BARS, ocho valoraciones independientes por participante y cuatro ítems por rasgo; la fiabilidad entre evaluadores va de ICC(1,8)=0,61 a 0,83. Son juicios agregados sobre conducta mostrada en una respuesta de entrevista, no autoinformes HEXACO ni mediciones independientes de rasgos estables. El método obtiene MSE de 0,1555, 0,1380, 0,1597 y 0,1077, con media 0,1402; MAE media 0,2982 y correlaciones de Pearson entre 0,4654 y 0,7258. Frente al Longformer de solo texto del propio artículo, el MSE mejora entre 14,33% y 48,83%. La ablación más informativa es más matizada: añadir las descripciones faciales a la misma arquitectura mejora H, X y C, pero empeora Agreeableness frente a la variante sin AUs, de MSE 0,1390 a 0,1597. Tampoco es el mejor resultado conocido de AVI-6: la publicación del reto informa medias 0,12284 y 0,13724 para sus dos primeras soluciones, aunque la comparación no es completamente controlada porque los documentos discrepan sin explicación entre 644/646 participantes y 450/452 casos de entrenamiento. La contribución defendible es arquitectónica: en este split y con estas características generadas, la fusión ayuda a predecir impresiones de evaluadores en tres de cuatro rasgos respecto a la ablación directa. No demuestra que las AUs revelen personalidad verdadera ni valida las interpretaciones post hoc de sinceridad, empatía o control. La cadena OpenFace→GPT-4o→resumen→Llama introduce varias capas de posible pérdida o invención; no hay evaluación humana de fidelidad semántica y los autores observan alucinaciones de dinámica temporal con ventanas de un solo fotograma. Las tablas ofrecen una sola estimación sin intervalos, distribución entre semillas o baseline ingenuo; no se publican número ni valores de las semillas, y GPT-4o usa parámetros por defecto sin seed. El pseudocódigo de simulated annealing tampoco actualiza el estado cuando acepta una solución peor. No hay código, datos derivados, predicciones, adaptadores ni entorno reproducible. Sobre todo, pese al encuadre de selección laboral, no se analizan errores por género, edad, raza, tono de piel, calidad de cámara u otras intersecciones; tampoco privacidad biométrica, consentimiento, impacto adverso, validez con desempeño laboral o uso real. Debe leerse como experimento de benchmark sobre personalidad aparente, no como evidencia para evaluar o contratar candidatos.

Research question

Does the prediction of apparent personality scores assigned by raters in AVI-6 improve when textual descriptions of facial action units generated by GPT-4o are fused with the transcripts using an LLM fine-tuned with LoRA?

Method

Supervised regression study per trait. OpenFace extracts 17 AUs; global pixel changes locate keyframes and form windows of seven frames. An LSTM and simulated annealing select AUs using validation MSE. GPT-4o verbalizes each window and fuses its descriptions. Meta-Llama-3.1-8B-Instruct quantized to 4 bits receives that summary, the transcript, and the HEXACO definition; LoRA and a linear layer predict a 1-5 score. Visual, textual, AU, and fusion models are compared using MSE, MAE, and Pearson correlation across four independent tasks.

Sample: The preprint uses 644 participants and declares a per-participant split of 450 training, 64 validation, and 130 test. Each person answers six questions; four distinct responses activate and label H, X, A, and C separately. The AVI-6 publication reports 307 men, 309 women, and 28 non-binary people recruited on Prolific, 12 trained raters, and eight independent ratings per participant. There is a documented discrepancy: the AVI-6 abstract says 646 participants and 3,876 videos and its split 452/64/130 sums to 646, while its text and demographics sum to 644; the preprint changes training to 450 without explaining the two cases.

Findings

  • The system achieves mean MSE 0.1402, mean MAE 0.2982, and mean correlation 0.6000 across four traits; individual correlations range from 0.4654 to 0.7258.
  • Compared to the text-only Longformer, it reduces MSE 14.33% in H, 48.83% in X, 22.21% in A, and 48.64% in C.
  • In the direct ablation without AU descriptions, the full model improves H, X, and C, but worsens Agreeableness 14.89% in MSE and 7.45% in MAE.
  • Qwen-3-VL-8B-Instruct achieves higher correlation in Agreeableness, 0.5589 versus 0.5154, although its absolute errors are much worse.
  • The seven-frame window obtains the best mean among 1, 3, 5, 7, and 9; five frames is better for Agreeableness.
  • The method does not surpass the two best published means of the AVI-6 challenge, 0.12284 and 0.13724, although the unexplained change in training count prevents treating it as a perfectly controlled comparison.
  • The corpus source confirms that the goal is to reproduce aggregated human judgments with moderate-to-high reliability, not to recover a self-reported personality score.

Limitations

  • The labels are apparent personality assessed on an elicited response; they are not contrasted with HEXACO self-report, longitudinal behavior, job performance, or hiring decisions.
  • Only four HEXACO domains are modeled; Emotionality and Openness are missing.
  • The preprint omits the number and training of raters and their ICCs, information needed to interpret the ceiling and meaning of the criterion.
  • The AVI-6 documents disagree between 644/646 participants and 450/452 training cases; the preprint does not document exclusions or split manifests.
  • Frame selection uses global changes without a facial mask, threshold, or minimum distance, so it may respond to lighting, camera, compression, or head movement.
  • There is no human evaluation verifying that the GPT-4o descriptions are faithful to the AUs; the study itself observes temporal hallucination with a one-frame window.
  • GPT-4o decisions use default service parameters and no seed; the quantity or values of training and search seeds are also not reported.
  • The results are single points without confidence intervals, cross-validation, distribution across runs, or comparison with a mean predictor.
  • It is unclear whether window size, subsets, and other decisions were locked using only validation before looking at the test set.
  • The simulated annealing pseudocode does not update the state when accepting a worse solution and the formulas do not specify zero-range or zero-overlap cases.
  • Psychological interpretations of the AUs are assigned after a loss-guided selection and are not validated as causal mechanisms or stable markers.
  • No code, generated descriptions, transcripts, predictions, seeds, adapters, checkpoints, dependency lock, or end-to-end run are published.
  • There is no fairness, error, or detection-failure analysis by demographic groups, skin tone, camera quality, or intersections.
  • Consent, biometric privacy, retention, right to challenge, human oversight, adverse impact, or candidate reactions are not studied.

What the study does not establish

  • It does not demonstrate that the system measures true or stable human personality; it predicts averages of observers on behavior in a simulated interview.
  • It does not establish convergent validity with self-reported HEXACO or predictive validity for job performance.
  • It does not prove that specific facial expressions cause or reveal sincerity, empathy, sociability, or control.
  • It does not demonstrate that adding AUs helps across all traits; it worsens Agreeableness compared to the ablation without AUs.
  • It does not establish state of the art in AVI-6; two challenge solutions publish lower mean MSE.
  • It does not evaluate the six HEXACO domains or allow generalization to Emotionality or Openness.
  • It does not demonstrate interpretability of the final predictor; the existence of readable intermediate prose does not explain the LoRA representation or the regression layer.
  • It does not demonstrate fairness or safety for hiring, because there are no per-group metrics, adverse impact, operational validation, or study with real applicants.
  • It does not allow exact reproduction of results or of the input features generated by GPT-4o.

Traceability

Scope: Full text

Version: arXiv:2606.29900v1

Consulted source: https://arxiv.org/pdf/2606.29900

Review: Codex 19-page full-text visual, TeX, supplement, eight-page source-dataset, construct, benchmark, semantic-fidelity, statistical, fairness, artifact, reproducibility and claim audit, 2026-07-16

Approval: Codex fidelity pass, 2026-07-16

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o (gpt-4o-2024-11-20) for AU semantic generation
  • Meta-Llama-3.1-8B-Instruct with 4-bit quantization and LoRA
  • Qwen-3-VL-8B-Instruct zero-shot baseline
  • Longformer text and AU-semantic baselines
  • ResNet video baseline
  • Vision Transformer video baseline
  • BERT fusion baseline
  • LSTM AU-selection and AU-only baseline

Instruments and metrics

  • AVI-6 four trait-eliciting asynchronous interview questions
  • Behavioral Anchored Response Scales with four items per trait
  • OpenFace dynamic facial action unit extraction
  • Whole-frame motion keyframes and seven-frame AU windows
  • Simulated annealing and Pareto/Utopia AU selection
  • Three published prompt templates for AU verbalization, iterative summary and trait embedding
  • HEXACO-PI-R trait definitions
  • Mean squared error, mean absolute error and Pearson correlation

Data used

  • AVI-6 mock asynchronous video interviews
  • Unreleased automatic transcripts
  • Unreleased GPT-4o local and global AU descriptions
  • Unreleased model predictions and checkpoints

Evidence and location

  • Preprint status, authors, and version date: Official arXiv record 2606.29900v1, checked 2026-07-16
  • Architecture, OpenFace, keyframes, AU selection, and Llama-LoRA prediction: arXiv v1, Sections III-IV and Algorithms 1-3
  • Complete prompts and AU definitions: arXiv v1, Supplementary Material, pages 17-19
  • MSE, MAE, correlations, ablations, and window sizes: arXiv v1, Sections V-VI, Tables I-V and Figures 5-8
  • Participants, demographics, split, BARS, raters, ICCs, and official AVI-6 results: ACM MM 2025 dataset/challenge paper, DOI 10.1145/3746027.3762016, Sections 3-4 and Tables 1-2
  • Consolidated audit of construct, split, SOTA, semantics, statistics, fairness, and reproducibility: reports/verification/article-289-arxiv-avi6-observer-label-split-sota-semantic-hallucination-fairness-reproducibility-and-claim-audit.json