Test hiring models for preference toward their own writing style
A controlled study found that LLM evaluators favoured résumés generated by the same model. Hiring teams need a model-origin invariance test before automated shortlisting affects candidates.
What happened
A revised research paper reports self-preference of 67% to 82% across tested models and a 23% to 60% shortlist advantage in simulations spanning 24 occupations when the applicant and evaluator used the same LLM.
Why it matters
The study tests a new failure mode that demographic fairness audits can miss: equally qualified candidates may be treated differently because their application resembles the evaluator model’s output.
The latest version of AI Self-preferencing in Algorithmic Hiring uses a controlled résumé correspondence experiment and simulated hiring pipelines across 24 occupations. The authors report self-preference of 67% to 82% across tested commercial and open-source models. In their simulations, equally qualified candidates using the same LLM as the evaluator were 23% to 60% more likely to be shortlisted than candidates submitting human-written résumés. They also report that simple interventions aimed at model self-recognition reduced the effect by more than half.
The paper is a preprint, and the pipeline is simulated. It does not estimate how often production recruiters use the same model as applicants, how prompts and retrieval alter the effect, or whether a final human decision corrects it. Financial Times reporting connects the result to a law firm’s internal comparison, but that anecdote is not a replication of the study.
This failure mode does not replace demographic, disability or accessibility testing. It cuts across them. A model-origin effect could compound an existing disadvantage or produce a new one among applicants who cannot, will not or do not know how to use a matching model. Report both tests separately and inspect their interaction; an average origin-invariance result can hide a larger gap in one group or occupation.
Add an origin-invariance gate
Build matched application sets with equivalent qualifications and several writing origins: human, the deployed model and alternative models. Randomise ordering, remove origin labels and run the exact production prompt, retrieval context and shortlist threshold. Measure selection-rate differences and rank instability by role family. Repeat after model, prompt or scoring changes.
Do not infer applicant competence from a model-like style. If a difference persists, test multiple evaluators, origin-blind instructions and structured job-evidence fields. Keep a human appeal route and record overrides. The immediate decision is whether the shortlisting system can demonstrate stable treatment when only the drafting model changes.