Corvus ISR Publishes Transparent Benchmark for Tracker Improvements

Corvus ISR tracker benchmark matrix (seed 1337) — The published matrix — every row reproducible. Source: corvusisr.com/benchmark

Corvus ISR, a leader in wide-area motion imagery (WAMI) exploitation, has taken a bold step in transparency by publishing its latest public tracker benchmark. This benchmark compares two distinct models on an identical fixed-seed synthetic scene with perfect ground truth, ensuring reproducibility and fairness. The seed 1337, combined with a consistent scenario setup—20 seconds of warm-up plus 120 seconds measured per row—provides a controlled environment for rigorous testing.

The first model, v1, employs a “greedy nearest-neighbour” approach, using a two-pass greedy association, constant-velocity prediction, and fixed 2-second coasting. This baseline is deliberately simple and serves as the published floor, yet it remains fully operational within the archived demo slices 1-2. In contrast, v2 introduces a “confirmed-track auction” method, utilizing a three-tier auction association, velocity-consistency gating, and noise-scaled reservation price to enhance tracking accuracy in the demo slice 3.

Results reveal significant improvements: for a baseline scenario with 150 movers at 2 fps, ID switches per minute dropped from 2,042 to 1,183, a reduction of 42.1%. In a denser scene with 400 movers, switches decreased from 14,032 to 8,040, a 42.7% improvement. Even under degraded conditions—such as 1 fps with jitter and 70% contrast—the models showed an 18% decrease in identity errors, demonstrating robust performance across various stress tests. It’s important to note that detection rate remains a sensor property, identical for both models, isolating the tracker’s impact.

Corvus emphasizes their commitment to honest testing by using a stricter ID switch metric than standard MOT-challenge definitions. Every change in ground-truth object identity counts as a switch, including fragmentations and re-acquisitions. This rigorous approach exposes the true performance and limitations of each model, highlighting that even state-of-the-art trackers still make thousands of identity errors per minute under stress. These results are published openly, not as marketing, but as measurement of real performance. The synthetic scenes with perfect ground truth ensure that these figures are measurable facts, not estimates or claims.

From an engineering perspective, v2 demonstrates impressive real-time performance, averaging around 1.2 milliseconds per sensor tick at a dense scene of 400 objects—well within a 10ms budget. Worst-case scenarios reach about 5ms, confirming suitability for live deployment. The entire benchmarking process is accessible: anyone can reproduce every row by visiting the live demo and pressing “Run benchmark,” with no signup or NDA required. This openness is further supported by the fact that v2 was built using an AI executor against a written acceptance contract, which was independently reviewed before deployment.

Everything in these tests is fully synthetic, with no real persons, vehicles, or locations involved. Each pixel is generated, ensuring a controlled environment suitable for precise measurement. This rigorous engineering discipline—focused on fixed-seed reproducibility, byte-identical harnesses, and honest metrics—sets a new standard for transparency in tracker benchmarking. Readers interested in seeing the results firsthand are encouraged to explore the public benchmark and reproduce it live. Why not try running the benchmark yourself and see the performance numbers in action?