Object Tracking
Object tracking associates observations of an object across time to estimate its continuing position or trajectory. The skill is combining detection, motion and appearance evidence while managing missed observations and identity changes. Tracking adds temporal association to perception; accurate detections in individual frames do not automatically yield reliable tracks.
What it is
Single-object tracking follows a designated target, while multi-object tracking maintains several identities and their trajectories. Tracking-by-detection detects objects in each frame, predicts their likely motion and matches new observations to existing tracks. SORT illustrates motion prediction and assignment from boxes; Deep SORT adds an appearance representation to support association. Track creation, termination and reappearance rules determine behavior during gaps. Camera motion, frame timing and coordinate changes affect motion estimates. Track IDs are local bookkeeping identities rather than verified real-world identities. Evaluation must therefore distinguish detection errors, trajectory fragmentation and incorrect switches between objects instead of using frame-level detection accuracy alone.
What the work involves
Define whether the application needs short-term continuity, counting or longer trajectories. Choose association features and motion assumptions consistent with the camera and frame rate. Annotate complete sequences and keep sequences separate across development and test sets. Evaluate identity switches, missed tracks and fragmentation alongside detection quality. Inspect crossings, occlusions and entry or exit behavior visually. Tune thresholds and gap rules on development sequences, then measure processing latency and stale-result handling. The useful result is a temporal pipeline whose track semantics, lifecycle and uncertainty are explicit enough for downstream counting or movement analysis.
Illustrative example
An illustrative warehouse camera counts carts crossing a doorway. The developer starts with detection-based tracks and reviews sequences where two carts overlap. A single cart changing track ID can be counted twice, so the counting rule uses a persistent crossing event rather than every newly created ID. Evaluation holds out full video sessions and checks both track identity and final counts. Long occlusions are reported as uncertain trajectories instead of inventing continuous movement through hidden regions.
Limits and common mistakes
Fast motion, occlusion, similar appearance and moving cameras can break association. A detector's false positive may become a persistent track, while reidentification can connect unrelated objects. Smooth trajectories may be wrong, and interpolated locations are estimates rather than observations. Cross-camera identity is a separate problem with additional assumptions. Distinguish visual tracking from facial or personal identification, and evaluate the downstream event logic as well as temporal association, especially when missed frames or variable timestamps occur.
Prerequisites
Related skills
- → is subcategory of: Object Detection
Sources and further reading
- Simple Online and Realtime Tracking
Motion-based tracking-by-detection and assignment.
- Simple Online and Realtime Tracking with a Deep Association Metric
Appearance-based association to reduce identity switches in multi-object tracking.
Last updated: 2026-10-10