Cross-Encoder Reranking

A bi-encoder represents query and document separately, enabling reusable document vectors and fast candidate search. A cross-encoder instead processes a query–document pair as one model input and predicts a relevance or relationship score. Because the document representation depends on the query, its score cannot ordinarily be precomputed once for every future search. The approach is therefore commonly applied to a limited candidate set. It is one implementation of search reranking rather than a synonym for all reranking, and its score meaning follows the model's training objective and calibration.

The practitioner selects a model appropriate for the language and relevance objective, then evaluates ranked candidates against labeled queries. Candidate recall is measured first because reranking cannot repair a missing document. Input-length limits, batching and candidate count are tested for latency and quality. Useful artifacts include the scoring configuration, before-and-after rankings and errors involving negation or near-identical entities. A held-out test checks whether improvements transfer beyond tuning examples. Application constraints and access filtering remain explicit rather than being inferred from the pair score.

Cross-encoders can misread a condition, favor familiar wording or truncate the decisive part of a long candidate. A high score is not factual verification, and scores may not be comparable across unrelated queries. More candidates increase opportunity and cost simultaneously. The technique needs task-specific ranking tests and an operational budget, with separate attention to candidate recall, model suitability and whether the reordered evidence actually improves supported answers.