Consensus Capture in Reference-Based UAV Video Stabilization: A Benchmark on Dynamic Urban Scenes
Abstract
Reference-frame stabilization registers every frame of a hovering-drone video onto one reference frame, keeping object positions comparable. The registration pipelines it relies on are selected on feature-matching benchmarks in which only the camera moves. Over a busy intersection, vehicles moving together supply mutually consistent correspondences that can draw the robust homography estimate toward their motion: consensus capture, which inlier statistics cannot flag. A two-part benchmark measures it and the object masking meant to prevent it. A static protocol scores ten pipelines under exactly known homographies on 4K aerial frames, tuned on a site-disjoint split at a matched resolution and feature budget. A dynamic protocol scores them at an offline and a real-time budget on hovers selected from the video alone to span the amount and coherence of moving foreground, plus a stationary control. It measures background alignment, object position error and the share of a moving group's displacement an estimate follows, against a pseudo-ground-truth reference audited with manually annotated ground control points, which also score every candidate. On static scenes, tuned AKAZE, RootSIFT, SIFT and BRISK lead, retuning the matcher thresholds raises the mean average accuracy (mAA) by at most 0.0062, and masking costs less than 0.005. Unmasked ORB is captured on 28 of 304 real-time dynamic pairs and on 6 offline; its real-time captured estimates have a median corner error of 18.8 px yet typical inlier counts and ratios. Masking removes capture for every pipeline and leaves the leading pipelines unchanged. Code, data and results are released.
Community
Reference-frame stabilization registers every frame of a hovering-drone video onto one reference frame, and its registration pipelines are usually picked on feature-matching benchmarks where only the camera moves. Over a busy intersection, vehicles moving together can pull the robust homography toward their own motion (consensus capture), while inlier counts and ratios still look normal.
We measure this with a two-part benchmark:
- Static: ten pipelines scored under exactly known homographies on 4K aerial frames, with tuning on a site-disjoint split.
- Dynamic: real hovers selected from the video alone, at an offline and a real-time budget, scored against a reference audited with manually annotated ground control points.
Unmasked ORB is captured on 28 of 304 real-time pairs; its captured estimates have a median corner error of 18.8 px with typical inlier statistics. Masking removes capture for every pipeline and leaves the leading pipelines unchanged.
Released: the code on GitHub, the benchmark data with all results on Zenodo, and Songdo Vision OBB, with 274,190 oriented boxes on 5,419 4K drone frames.
Consensus capture is a sneaky failure: vehicles moving together agree with each other, so the homography looks healthy while the background drifts. Inlier counts staying normal while the median corner error sits at 18.8 px is the part that worries me. Did masking cost anything on the dynamic hovers, or only on static scenes?
Thanks! Masking comes at no measurable cost on the dynamic hovers either. In both the offline and the real-time regime, SIFT, RootSIFT and AKAZE remain the leading pipelines under every masking mode (none, horizontal boxes, oriented boxes), while masking removes capture for every pipeline. ORB, the pipeline that gets captured, gains the most: its real-time mAA rises from 0.584 unmasked to 0.679 with oriented-box masks. On static scenes, masking costs every pipeline less than 0.005 mAA. The Part II masking results in the paper and the per-pair records on Zenodo give the full breakdown.
Models citing this paper 0
No model linking this paper
Datasets citing this paper 2
rfonod/stabilo-benchmark
Spaces citing this paper 0
No Space linking this paper