Method

Motion-Compensated Multi-Sensor Fusion with Hierarchical Association for 3D Multi-Target Tracking [MCMSF-HA-3DMTT]


Submitted on 6 Aug. 2024 10:13 by
(University of South China)

Running time:0.01 s
Environment:1 core @ 2.5 Ghz (C/C++)

Method Description:
We present MCMSF-HA-3DMTT, a novel framework that
achieves robust 3D multi-target tracking through
motion-compensated sensor fusion and hierarchical
association. First, high-precision spatial and
feature alignment is performed between LiDAR point
clouds and a forward-facing monocular camera, and
a dynamic confidence-weighted fusion strategy is
applied to enhance the robustness of the 3D
detector. Next, a coupled-state Kalman filter
recursively estimates target states while
compensating for ego-vehicle motion to eliminate
platform-induced interference. For data
association, we introduce a multi-level spatial
indexing structure; at each level, geometric and
appearance cues are jointly fused, and targets are
matched via a stage-wise gating and assignment
protocol. Extensive experiments on public
benchmarks demonstrate that MCMSF-HA-3DMTT
significantly improves tracking accuracy and
stability in complex driving scenarios.
Parameters:
\
Latex Bibtex:
@inproceedings{zhang2026motion,
title = {Motion-Compensated Multi-Sensor
Fusion with Hierarchical Association for 3D Multi-
Target Tracking},
author = {Zhang, Mengyao and Jiang, Chao and
Zhang, Mingyue},
booktitle = {Proceedings of the AAAI Conference
on Artificial Intelligence (AAAI)},
year = {2026},
note = {Submitted for review; under
consideration},
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 65.14 % 64.64 % 66.38 % 72.51 % 73.25 % 69.65 % 83.33 % 81.50 %
PEDESTRIAN 51.26 % 49.71 % 53.25 % 58.58 % 62.11 % 58.65 % 70.14 % 75.79 %

Benchmark TP FP FN
CAR 30925 3467 3121
PEDESTRIAN 17840 5310 3993

Benchmark MOTA MOTP MODA IDSW sMOTA
CAR 80.22 % 78.95 % 80.84 % 213 61.30 %
PEDESTRIAN 58.40 % 71.18 % 59.81 % 328 36.19 %

Benchmark MT rate PT rate ML rate FRAG
CAR 73.85 % 22.92 % 3.23 % 324
PEDESTRIAN 52.92 % 36.77 % 10.31 % 1078

Benchmark # Dets # Tracks
CAR 34046 1280
PEDESTRIAN 21833 737

This table as LaTeX


This figure as: png pdf

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker