Glossary · term

Generative Test-Set Contamination

A paper (arXiv:2601.04301, January 2026, 11 authors including Stella Biderman) showing that even a single copy of a generative benchmark in the pretraining data allows a model to achieve a loss lower than the "irreducible error" of training on an uncontaminated corpus.

Safety2026Wave 3 · 2025–26Maturity: 2/5

Maturity rationale

single source, early stage

References

'Quantifying the Effect of Test Set Contamination on Generative Evaluations' (Sc(arxiv)

Author: Społeczność / Anonimowi

Cite or link to this resource · CC BY 4.0

Free to quote, link, and reuse with attribution

Referencing this analysis in documentation, research, or an engineering write-up? Copy a formatted link or citation below:

Show citation text & canonical URL

Markdown:

[Generative Test-Set Contamination — AI Glossary — Skills Intelligence](https://www.skillsintelligence.tools/glossary/term/generative-test-set-contamination)

APA:

Skills Intelligence. (2026). Generative Test-Set Contamination — AI Glossary. Skills Intelligence AI Glossary. Retrieved from https://www.skillsintelligence.tools/glossary/term/generative-test-set-contamination