Character-LLM proposes specializing a separate LLaMA-7B model for each character through synthetic experiences rather than relying only on role-play instructions. For nine historical or fictional figures, Cleopatra VII, Voldemort, Spartacus, Hermione Granger, Isaac Newton, Julius Caesar, Beethoven, Socrates, and Martin Luther King, the pipeline splits Wikipedia profiles into chunks, asks GPT-3.5-turbo to imagine 20 plausible scenes per chunk, and expands each scene into a script of at least 1,200 words containing dialogue, actions, and thoughts. It adds fewer than one hundred “protective experiences” per character in which the simulacrum learns to appear confused when asked about knowledge incompatible with its era or identity. Each agent is fine-tuned separately for ten epochs on roughly 1,400-2,200 scenes, 599,000 to 1,038,000 words per character, with a 2,048-token context, effective batch size 64, and eight 80-GB A100 GPUs. The paper explicitly lets the model overfit despite rising development perplexity and manually chooses five- or ten-epoch checkpoints using only ten held-out questions. Evaluation contains 857 single-turn questions and 450 multi-turn scenarios, 50 per character and five turns in the code, and compares Character-LLM with prompt-guided Alpaca-7B, Vicuna-7B, and GPT-3.5-turbo. GPT-3.5 helps generate questions, conducts interviews, and scores five 1-7 dimensions: memorization, values, personality, avoidance of anachronistic knowledge, and stability. Figure 4 suggests that trained agents mainly outperform Alpaca/Vicuna in personality, memorization, stability, and rejection of anachronistic questions, remain weaker on values, and are comparable to ChatGPT. However, the axes are truncated at approximately 6.4-7, exact values, dispersion, and statistical tests are not reported, and there is no human evaluation. Qualitative cases do illustrate the mechanism: a trained Beethoven recalls concrete details about his parents and, with protective scenes, avoids writing Python quicksort. This supports the claim that SFT on synthetic scripts can memorize a textual role representation and teach contextual abstention; it does not show that an agent possesses the person's real personality, emotions, autobiographical memory, or values. The training source is not “actual experience”: GPT invents dialogue, thoughts, and details from secondary profiles, and prompts instruct it to forget that it is a model, treat the character as real, and disregard moral, legal, and social constraints. The same GPT family acts as generator, baseline, interviewer, and judge, creating circularity and possible self-evaluation bias; “personality” is reduced to narrative resemblance to the profile, without a psychological instrument. The artifact release is valuable: the Apache-2.0 repository links nine weight deltas and a CC BY-NC 4.0 dataset and contains profiles, questions, and the expected 3,428 single-turn responses plus 1,800 multi-turn interviews. Yet the audited official commit has serious limitations: no root requirements, project tests, training seeds, score outputs, or Figure 4 script/data; the Hugging Face viewer fails because schemas are mixed; scoring scripts keep DEBUG=True and process Beethoven only; evaluation uses only the first profile paragraph; the implemented baseline prompt omits the profile the paper says it provides; documentation starts the server on port 28001 while clients call 8000; and one call passes temperature=0.2 positionally as n, which should be an integer. The README uses a cosine scheduler while the paper describes linear decay, shows temp-0.2 filenames although code defaults to 0.7, asks users to place keys in a tracked file, and relies on the legacy openai.ChatCompletion interface without pinned versions. Character-LLM is therefore an influential and partially open precedent for role-play fine-tuning, but its quantitative results remain exploratory and unreproduced, and its code needs repair before the full pipeline can run.
Research question
Can a 7B LLM specialized through reconstructed narrative experiences represent a specific character with greater credibility, memory, personality, values, stability, and knowledge boundaries than agents guided solely by prompt?