BLEU

BLEU counts candidate n-grams that appear in reference translations, clipping matches so repetition cannot receive unlimited credit. It combines precision across n-gram lengths and applies a brevity penalty when candidate output is too short relative to references. Corpus-level aggregation is central to its original use; sentence-level values depend strongly on smoothing and implementation. BLEU differs from ROUGE measures that emphasize overlap recall and from embedding-based semantic metrics. Tokenization, casing, reference selection and score scaling affect comparisons, so a reported number needs a clear calculation protocol rather than just the metric name.

The practitioner uses a documented implementation and records tokenization, smoothing, reference handling and whether scoring is sentence or corpus based. It checks examples where valid paraphrases receive low overlap and where lexical similarity hides a meaning error. Useful artifacts include the scoring configuration, comparable system outputs and human-reviewed samples. For translation, BLEU can complement expert adequacy and fluency review. For open-ended generation, task-specific factual and completeness checks are needed. Comparisons should not mix numbers produced by different preprocessing conventions or reference collections.

Valid wording can differ from a limited reference set, and high overlap can coexist with a critical factual or grammatical error. Short outputs make sentence scores unstable, while tokenization differences can distort comparisons. BLEU should be interpreted within its reference and calculation setting. It is a measurement of lexical overlap designed for a particular evaluation tradition, not a general correctness score for language-model answers or a replacement for task-relevant human assessment.