Our paper, “Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior,” has been selected for Oral Presentation at CTB
@icmlconf
* Paper:
arxiv.org/abs/2606.12730
* Website:
psychology-of-ai.github.io/p… * Code:
github.com/psychology-of-AI/…
A central question in AI evaluation is whether we can use low-cost self-report probes to anticipate how LLMs will actually behave in tasks.
In our earlier work, “The Personality Illusion,” we found that LLMs can give coherent personality self-reports that do not reliably predict behavior. This paper asks a follow-up question: When do self-reports actually track behavior, and what are the failure modes where they don't?
Across 11 LLMs, 4 behavioral tasks, and a 2 × 2 × 2 experimental design, we find that self-report–behavior coherence exists, but it is selective:
1) The instrument matters. Broad Big Five personality traits do not predict task behavior well. But a more behavior-specific framework, the Theory of Planned Behavior, can recover much stronger coherence under favorable conditions.
2) Context matters. When self-reports and behavior happen in the same conversation, coherence can reach human-level intention–behavior baselines. But when they happen in separate conversations, coherence often collapses.
3) The task matters. Coherence survives better for behaviors anchored outside the immediate prompt, such as implicit bias and aspects of honesty. It collapses for behaviors strongly shaped by the local context, such as sycophancy.
4) Personas are not a fix. Persona prompting makes models’ self-reports more stable across conversations, but it does not reliably bring behavior into alignment. This is especially important for persona-customized AI systems: changing what a model says about itself does not necessarily change what it does.
The takeaway: LLM self-reports should not be treated as context-free behavioral diagnostics. If we want to use psychometric probes for AI safety, deployment, or model evaluation, we need task-specific instruments, behaviorally grounded validation, and careful separation between what a model says and what it actually does.
Huge thanks to my co-authors
@RKocielnik Pengrui Han, Peiyang Song, Myrl G. Marmarelis, Ramit Debnath, Dean Mobbs, and R. Michael Alvarez, and to the
@Caltech Linde Center for Science, Society, and Policy
@CaltechLCSSP
