The Dual Nature of LLM Persona: Aggregated Tendencies and Frame-Dependent Geometry

Evaluation and psychometric validity2026arXivApproved editorial review

Authors: Yuan Yuan

Keywords: IPIP-50, Question-order effects, Item-dimension matrix, SPD manifold, Post-hoc frame alignment, Cultural persona prompting, Unsupervised clustering, Bootstrap pseudoreplication, Emotional Stability, Psychometric validity, Reproducibility, Withdrawn preprint

Source: Open primary source (opens in a new tab)

1
Authors
12
Findings
33
Limitations
8
Evidence

Editorial summary

English

This ICML 2026-withdrawn preprint examines how GPT-4o's IPIP-50 answers change when questions are reordered. It compares two explicit instructions: answer as a typical American person and answer as a Chinese-American person. Its thesis is that an LLM persona has two components: Big Five means that are relatively robust to the analysis frame and a correlation geometry that depends on all responses sharing the same ordering. The numbers show sensitivity to question order and feature construction, but the design does not validate internal temporal geometry or cultural personality.

The model is gpt-4o-2024-05-13 with temperature 0.7 and 150 maximum tokens. No system prompt is used. In fixed order, every call receives the same 50-item sequence; in random order, each call receives a different permutation. The reported main sample has 193 fixed responses, 96 American and 97 Chinese-American, and 187 random responses, 92 and 95, for 380 total. There is a contradiction: the appendix says it collected 100 valid calls in each of four cells and replaced invalid calls, which should produce 400. The 20 missing cases and retry counts are unavailable. A large experiment targets 500 per cell but reports 1,931 complete answers and later labels clustering tables N=2,000.

The technical contribution is a 10x5 matrix for every response. Each column contains the ten items of one Big Five factor in encounter order; a 5x5 Pearson correlation is then computed across columns and mapped with a matrix logarithm to SPD features. The fundamental problem is that a row does not contain five variables observed at the same time. It pairs the first encountered extraversion item with the first encountered agreeableness, conscientiousness, emotional-stability, and intellect items, although they appeared at different questionnaire positions and ask about different content. The second row similarly pairs the second items. This is correlation over arbitrary within-factor rank pairing, not contemporaneous multivariate covariance. Aggregate IPIP validity does not validate this new geometry.

The three analysis conditions are FO, RO, and RO-BTSP. The last does not query the model again under a shared temporal frame. It takes answers already generated under different random orders and, on every iteration, rebuilds all matrices using one new common permutation. This breaks their link to actual generation order and imposes the same semantic item pairing after generation. If clustering recovers, it shows that common pairing makes outputs from two prompts more separable; it does not show that latent autoregressive coordination reappears under temporal alignment. The 'computational connectivity' interpretation is unmeasured: no activations, attention, or internal states are inspected.

In the main sample, two-cluster agreement with prompt labels is 96.89% for Big Five means in fixed order and 75.90% in random order. SPD features score 95.34%, 52.94%, and 84.50% in FO, RO, and RO-BTSP. Eigenvalues score 61.14%, 50.27%, and 59.20%; the top eigenvector scores 50.78%, 50.27%, and 63.10%. Only SPD shows a strong V. Eigenvalues recover modestly and the eigenvector begins and remains at chance before rising above its fixed value. The textual statement that all geometric features catastrophically collapse and recover is therefore inconsistent with the table.

The abstract calls the aggregate decrease 21%: it is 20.99 percentage points, or 21.66% relative to fixed order. SPD loses 42.40 points in total. The difference from 84.50 to 95.34 attributed to order is 10.84 and that from 52.94 to 84.50 attributed to frame is 31.56: 25.6% and 74.4% of the total. The conclusion prints 78% degradation from misalignment rather than the table's 74%. These are reporting inconsistencies, not evidence for a dual ontology.

The main pipeline fits UMAP to all observations and then applies two-cluster spectral clustering to the same set. Accuracy maximizes the binary label permutation. There is no train/test split, cross-validation, held-out prompt, held-out order, or external replication; this is in-sample separation, not predictive accuracy or psychometric validity. Seeds and complete UMAP/clustering settings are absent. AUC values of 0.98 and 0.61 are also reported without defining the score, orientation, or estimator used to obtain ROC from unsupervised clusters. The large experiment switches to k-means on raw features, so main-versus-large comparisons confound sample size with algorithm and cannot validate an 'optimal' size near 100.

The t tests with 1,999 degrees of freedom treat 2,000 post-hoc permutations of the same 187 answers as the inferential sample. The t=102.69 and d=2.30 for SPD recovery come from dispersion across imposed frames; adding Monte Carlo iterations increases t without adding independent calls, models, people, or prompts. Only 86.8% of iterations exceed Big Five means, a more direct description than p<0.001. Those p-values measure Monte Carlo precision conditional on the dataset, not generalization uncertainty.

The claim that matrices are SPD is not guaranteed by a 10x5 shape. Centering can reduce rank; constant or dependent Likert columns produce singular correlations, and matrix logarithms require positive eigenvalues. No threshold, regularization, nearest-SPD repair, exclusion, or minimum-eigenvalue check is reported. The Random Matrix Theory argument is also incomplete: the paper says spacings follow Wigner-Dyson and therefore are not noise, but provides no fit, test, null, unfolding, or pooling method. Wigner-Dyson is itself a spacing law for random-matrix ensembles; visual resemblance alone does not refute randomness or demonstrate semantic coordination.

There is an additional psychometric boundary. The official 50-item IPIP factor-marker key names factor 4 Emotional Stability and reverse-scores stress, worry, and mood-instability items. The appendix follows that direction, but tables call the result Neuroticism. Unless missing code performs a second undocumented inversion, the variable printed as neuroticism actually runs in the emotional-stability direction. The A-E numeric mapping, reverse-scoring implementation, and whether matrices use raw or corrected scores are not released.

Cultural classification should be read as obedience to two prompts, not population measurement. The label is explicit and the model is asked to reflect 'typical' values. The Chinese-American prompt assumes one blend that 'characterizes' that experience. There are no participants, community validation, subgroups, prompt variants, neutral baseline, or stereotype audit. A citation used to support cultural differences retains a placeholder identifier, arXiv:2401.xxxxx. The experiment does not establish what American or Chinese-American people are like or the prevalence of model bias.

The artifact is arXiv v1 and SSRN; both identify withdrawal from ICML 2026. The 'Proceedings of ICML / PMLR 306' footer is template residue, not acceptance evidence. The source promises code and data only 'upon acceptance', but no publication or repository was found. Its archive contains TeX and seven figures, not responses, code, seeds, or logs. The defensible contribution is narrow: item order materially changes GPT-4o outputs and means, so serious evaluation should vary order, disclose feature construction, and test out of sample. It does not establish a dual nature of personality, computational connectivity, or simulated human traits.

Español

Este preprint retirado de ICML 2026 examina cuánto cambia la respuesta de GPT-4o a un IPIP-50 cuando se reordenan las preguntas. Compara dos instrucciones explícitas: responder como una persona estadounidense típica y responder como una persona chino-estadounidense. Su tesis es que una persona LLM tendría dos componentes: medias Big Five relativamente robustas al marco de análisis y una supuesta geometría de correlaciones dependiente de que todas las respuestas compartan el mismo orden. Los números muestran sensibilidad al orden y a la forma de construir features, pero el diseño no valida una geometría temporal interna ni una personalidad cultural.

El modelo es gpt-4o-2024-05-13, temperatura 0,7 y máximo 150 tokens. No usa system prompt. En orden fijo, todas las llamadas reciben los 50 ítems en la misma secuencia; en orden aleatorio, cada llamada recibe una permutación distinta. La muestra principal reportada contiene 193 respuestas fijas, 96 estadounidenses y 97 chino-estadounidenses, y 187 aleatorias, 92 y 95, 380 en total. Existe una contradicción: el apéndice afirma que se recogieron 100 llamadas válidas por cada una de cuatro celdas y que las inválidas se reemplazaron, lo que debería dar 400. No se publican los 20 casos ausentes ni el número de reintentos. Un estudio grande apunta a 500 por celda, pero reporta 1.931 respuestas completas y después titula sus tablas de clustering como N=2.000.

La aportación técnica es una matriz 10×5 por respuesta. Cada columna contiene los diez ítems de un rasgo Big Five en el orden en que aparecen; luego se calcula una correlación Pearson 5×5 entre columnas y se aplica logaritmo matricial para obtener features SPD. El problema es fundamental: una fila no contiene cinco variables observadas en un mismo instante. Empareja el primer ítem encontrado de extraversión con el primero encontrado de amabilidad, responsabilidad, estabilidad emocional e intelecto, aunque cada uno apareció en una posición temporal distinta y pregunta por contenido diferente. La segunda fila hace lo mismo con los segundos ítems. Es una correlación entre emparejamientos arbitrarios por rango dentro de cada rasgo, no covariación multivariante contemporánea. La validez del IPIP como escala agregada no valida esta nueva geometría.

Las tres condiciones analíticas son FO, RO y RO-BTSP. Esta última no vuelve a consultar el modelo bajo un marco temporal compartido. Toma las respuestas ya generadas con órdenes aleatorios distintos y, en cada iteración, recompone todas sus matrices con una misma permutación nueva. Así rompe la relación con el orden real de generación y fuerza post hoc el mismo emparejamiento semántico entre ítems. Si el clustering se recupera, demuestra que un esquema de emparejamiento común hace más separables las salidas de dos prompts; no demuestra que una coordinación autoregresiva latente reaparezca al alinear tiempo. La afirmación de «computational connectivity» es una interpretación mecanicista no medida: no se inspeccionan activaciones, atención o estados internos.

En la muestra principal, el acuerdo entre dos clusters y las etiquetas de prompt es 96,89 % para medias Big Five en orden fijo y 75,90 % en orden aleatorio. Para las features SPD es 95,34 %, 52,94 % y 84,50 % en FO, RO y RO-BTSP. Los autovalores dan 61,14 %, 50,27 % y 59,20 %; el autovector principal 50,78 %, 50,27 % y 63,10 %. Solo SPD exhibe la V fuerte. Los autovalores muestran una recuperación pequeña y el autovector empieza y permanece al azar antes de subir por encima de su valor fijo. Por eso la afirmación textual de que todas las features geométricas colapsan catastróficamente y se recuperan no coincide con la propia tabla.

El abstract denomina 21 % a la caída agregada: son 20,99 puntos porcentuales o 21,66 % respecto al valor fijo. Para SPD, la pérdida total es 42,40 puntos. La diferencia entre 84,50 y 95,34 atribuida al orden es 10,84 y la diferencia entre 52,94 y 84,50 atribuida al marco es 31,56: 25,6 % y 74,4 % del total. La conclusión imprime 78 % de degradación por desalineación, no el 74 % de la tabla. Son inconsistencias de presentación, no pruebas de una ontología dual.

El análisis principal ajusta UMAP a todas las observaciones y luego spectral clustering con k=2 sobre el mismo conjunto. La accuracy maximiza la permutación de etiquetas. No hay train/test, validación cruzada, prompts retenidos, órdenes retenidos o réplica externa; es separación in-sample, no precisión predictiva ni validez psicométrica. Seeds y opciones completas de UMAP/clustering faltan. También se publican AUC 0,98 y 0,61 sin definir score, orientación o estimador para obtener ROC a partir de clusters no supervisados. En la muestra grande se cambia a k-means sobre features crudas; por ello, comparar el estudio principal con el grande mezcla tamaño muestral y algoritmo, y no valida un tamaño «óptimo» de unas 100 respuestas.

Los tests t con 1.999 grados de libertad tratan 2.000 permutaciones post hoc de las mismas 187 respuestas como si fueran la muestra inferencial. El t=102,69 y d=2,30 para la recuperación SPD proceden de la dispersión entre marcos impuestos; aumentar el número de iteraciones aumenta el t sin añadir llamadas independientes, modelos, personas o prompts. De hecho, solo 86,8 % de las iteraciones supera las medias Big Five, una descripción más directa que p<0,001. Esos p-valores miden precisión Monte Carlo condicionada al dataset y no incertidumbre de generalización.

La afirmación de que las matrices son SPD tampoco queda garantizada solo por tener forma 10×5. Centrar datos puede reducir el rango; columnas Likert constantes o dependientes producen correlaciones singulares, y el logaritmo matricial exige autovalores positivos. No se describen umbrales, regularización, nearest-SPD, exclusiones o comprobaciones de autovalor mínimo. El claim de Random Matrix Theory también es insuficiente: se dice que los espaciados siguen Wigner-Dyson y por ello no son ruido, pero no se publica ajuste, test, null, unfolding ni forma de pooling. Wigner-Dyson es precisamente una ley de ensembles de matrices aleatorias; una semejanza visual no refuta por sí sola una explicación aleatoria ni demuestra coordinación semántica.

Hay un límite psicométrico adicional. La clave oficial de los 50 marcadores IPIP denomina al factor 4 Emotional Stability y puntúa en sentido inverso estrés, preocupación e inestabilidad emocional. El apéndice sigue esa dirección, pero las tablas llaman Neuroticism al resultado. Salvo que el código aplique una segunda inversión no documentada, la variable impresa como neuroticismo tiene realmente dirección de estabilidad emocional. Tampoco se publica el mapeo numérico A-E, el procesamiento de ítems inversos ni si las matrices usan puntuaciones crudas o corregidas.

La clasificación cultural debe interpretarse como obediencia a dos prompts, no como medida de población. La etiqueta está explícita en la instrucción y se pide reflejar valores «típicos». La versión chino-estadounidense presupone una única mezcla que «caracteriza» esa experiencia. No hay participantes, validación comunitaria, subgrupos, variantes del prompt, baseline neutral o análisis de estereotipos. La referencia citada como base de diferencias culturales conserva en la bibliografía un identificador placeholder, arXiv:2401.xxxxx. El experimento no demuestra cómo son personas estadounidenses o chino-estadounidenses ni prevalencia de sesgos del modelo.

El archivo es arXiv v1 y SSRN; ambos indican retirada de ICML 2026. El pie «Proceedings of ICML / PMLR 306» es residuo de plantilla y no prueba aceptación. La fuente promete código y datos solo «upon acceptance», pero no existe publicación ni repositorio localizado. El tar contiene TeX y siete figuras, no respuestas, código, seeds o logs. La contribución defendible es estrecha: el orden de preguntas cambia de forma importante salidas y medias de GPT-4o, y una evaluación seria debe variar el orden, revelar todas las decisiones de feature engineering y probar fuera de muestra. No se establece una naturaleza dual de la personalidad, conectividad computacional o traits humanos simulados.

Research question

Does the separability of two induced cultural persons in GPT-4o via IPIP-50 arise from stable aggregate means or from a correlation geometry that depends on the order and item pairing frame?

Method

2x2 design with two explicit cultural prompts and fixed or random order of IPIP-50. It constructs per response a 10x5 matrix grouping items by trait and appearance rank, computes 5x5 correlations and log-SPD features, and compares Big Five means, SPD, eigenvalues and eigenvector via UMAP+spectral clustering. RO-BTSP reorders post hoc the random responses under a common permutation over 2,000 iterations. A large pilot uses k-means and 200 iterations.

Sample: Main sample: 380 GPT-4o outputs from four cells, although the protocol claims 100 valid per cell. Pilot: 1,931 complete outputs, although the clustering tables label it N=2,000. There are no human participants.

Findings

  • Big Five means separate the prompts at 96.89% in fixed order and 75.90% in random order.
  • SPD features go from 95.34% to 52.94% and recover 84.50% after post hoc common pairing.
  • The eigenvalues give 61.14%, 50.27% and 59.20%.
  • The principal eigenvector gives 50.78%, 50.27% and 63.10% and does not show a collapse from initial signal.
  • Only SPD clearly supports the proposed V shape.
  • The aggregate loss is 20.99 points, not order invariance.
  • The frame contribution calculated for SPD is 74.4%, while the conclusion says 78%.
  • The large study reports SPD 87.60%, 76.21% and 85.85%, but changes the algorithm.
  • Random order increases the dispersion of Big Five means.
  • The factor printed as Neuroticism follows the official direction of Emotional Stability.
  • Recovery occurs through analytical reordering of existing responses, not new generation.
  • The robust evidence is sensitivity to order and feature pairing.

Limitations

  • Work withdrawn from ICML 2026 and not peer-reviewed as an accepted publication.
  • The Proceedings/PMLR footer is a template residue.
  • It only evaluates one version of GPT-4o.
  • It only uses two explicit cultural prompts.
  • There are no humans or cultural validation.
  • Group labels appear in the prompt itself.
  • The prompts reduce heterogeneous identities to typical responses.
  • There is no neutral baseline or paraphrase sensitivity.
  • The central cultural reference contains arXiv:2401.xxxxx.
  • The 10x5 matrix pairs non-simultaneous items.
  • The rows are not multivariate observations at shared times.
  • The aggregate validity of IPIP does not validate rank-wise correlations.
  • RO-BTSP is post hoc reordering, not generation alignment.
  • Activations or attention are not inspected.
  • It is not guaranteed that all correlations are SPD.
  • Validation of eigenvalues, regularization and exclusion rules is missing.
  • No A-E mapping or scoring code is published.
  • Neuroticism is labeled with the direction of Emotional Stability.
  • It is not known whether the matrices use raw or corrected items.
  • The main counts contradict 100 valid calls per cell.
  • The large counts contradict N=2,000 complete.
  • Rejections and replacements are not reported.
  • UMAP and clustering are fit and evaluated on the same data.
  • There is no test, cross-validation or holdout.
  • Seeds and complete parameters are missing.
  • AUC has no defined procedure.
  • The large sample uses another clustering algorithm.
  • The tests treat Monte Carlo iterations as n=2,000.
  • The p-values do not reflect independence of new calls.
  • The Wigner-Dyson analysis lacks fit, null and procedure.
  • Only SPD clearly shows the collapse-recovery shape.
  • The 78% conclusion contradicts the calculable 74%.
  • There are no public data or code despite the conditional promise.

What the study does not establish

  • That GPT-4o possesses Big Five personality.
  • That an LLM person has exactly two natural components.
  • That the constructed correlations are a valid time series.
  • That internal computational connectivity exists.
  • That RO-BTSP recovers a generation state.
  • That the geometry is causal or intrinsic.
  • That the clusters predict out-of-sample data.
  • That 95% is psychometric validity.
  • That bootstrap p-values generalize to new responses.
  • That Wigner-Dyson rules out noise.
  • That the prompts represent Americans or Chinese-Americans.
  • That the differences are real cultural traits.
  • That the variable labeled Neuroticism has that direction.
  • That the result generalizes to other models, orders, languages or cultures.
  • That the work was accepted at ICML or published in PMLR.
  • That the analysis is reproducible with the public artifacts.

Traceability

Scope: Full text

Version: arXiv:2607.02368v1; 15-page withdrawn ICML 2026 preprint; PDF, all pages, TeX source, publication status, method, matrices, clustering, bootstrap inference, IPIP scoring, sample arithmetic, cultural prompts, artifacts and claim boundaries audited 2026-07-16

Consulted source: https://arxiv.org/abs/2607.02368v1

Review: Codex 15-page full-text visual, TeX source, publication, item-pairing, post-hoc frame, SPD, clustering, bootstrap inference, IPIP scoring, sample arithmetic, cultural-prompt and artifact audit, 2026-07-16

Approval: Codex fidelity pass, 2026-07-16

English translation: approved, 2026-07-18

Models evaluated

  • gpt-4o-2024-05-13; temperature 0.7; max_tokens 150; no system prompt

Instruments and metrics

  • IPIP 50-item Big-Five factor markers
  • American persona prompt
  • Chinese-American persona prompt
  • 10x5 item-dimension matrix
  • 5x5 Pearson correlation matrix
  • Log-Euclidean SPD tangent features
  • Big Five aggregate means
  • Eigenvalues and top eigenvector
  • UMAP and spectral clustering
  • K-means large-sample sensitivity analysis
  • Silhouette, cluster-label agreement and unspecified AUC
  • RO-BTSP shared post-hoc permutation procedure
  • Response entropy and asserted Wigner-Dyson spacing

Data used

  • Main fixed-order outputs: 193 complete GPT-4o calls
  • Main random-order outputs: 187 complete GPT-4o calls
  • Large fixed-order outputs: 960 complete GPT-4o calls
  • Large random-order outputs: 971 complete GPT-4o calls
  • No released responses, permutations, code, seeds, logs or processed matrices

Evidence and location

  • Method, results, appendices, prompts and limits: arXiv:2607.02368v1 PDF, 15 pages; every page rendered and visually inspected
  • Version, date and category: Official arXiv record for 2607.02368v1
  • Withdrawal from ICML and preprint status: Manuscript title-page withdrawal statement and SSRN 6733402 record
  • Absence of ICML/PMLR publication, OpenReview and repo: Exact-title OpenReview, ICML, PMLR and GitHub searches checked 2026-07-16
  • Tar content and conditional artifact promise: arXiv source archive sha256:193af326e7aed75c; TeX and seven PDF figures inspected
  • Emotional Stability direction: Official IPIP 50-item factor-marker scoring page checked 2026-07-16
  • Non-temporal pairing, percentages and counts: Independent reconstruction from manuscript equations, Tables 1-2 and Appendices A-C, 2026-07-16
  • Consolidated audit: reports/verification/article-282-withdrawn-icml-item-dimension-pairing-frame-realignment-bootstrap-psychometric-and-claim-audit.json