Method

SAM2-based Multi-object Tracking and Segmentation using Zero-shot Learning [Seg2Track-SAM2]
github.com/hcmr-lab/Seg2Track-SAM2

Submitted on 9 Sep. 2025 16:24 by
Diogo Mendonça (Universidade de Coimbra)

Running time:1 s
Environment:GPU @ 1.5 Ghz (Python)

Method Description:
This method extends SAM2 to multi-object tracking
and segmentation in a zero-shot setting. Objects are
initialized with a detector and refined over time
through object reinforcement, ensuring consistent
masks across frames without extra training.
Parameters:
\detection_threshold=0.5
\removal_threshold=0.1
Latex Bibtex:
@misc{mendonça2025seg2tracksam2sam2basedmultiobjecttracking,
title={Seg2Track-SAM2: SAM2-based Multi-object Tracking and Segmentation for Zero-shot Generalization},
author={Diogo Mendonça and Tiago Barros and Cristiano Premebida and Urbano J. Nunes},
year={2025},
eprint={2509.11772},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2509.11772},
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 74.13 % 71.03 % 78.15 % 78.81 % 81.90 % 81.28 % 91.14 % 87.69 %
PEDESTRIAN 60.00 % 56.61 % 65.86 % 65.65 % 69.35 % 71.76 % 79.98 % 80.40 %

Benchmark TP FP FN
CAR 32623 4137 2750
PEDESTRIAN 16887 3810 2706

Benchmark MOTSA MOTSP MODSA IDSW sMOTSA
CAR 81.01 % 86.19 % 81.27 % 95 68.75 %
PEDESTRIAN 68.14 % 77.41 % 68.52 % 79 49.71 %

Benchmark MT rate PT rate ML rate FRAG
CAR 71.62 % 25.83 % 2.55 % 378
PEDESTRIAN 56.67 % 29.26 % 14.07 % 399

Benchmark # Dets # Tracks
CAR 35373 803
PEDESTRIAN 19593 310

This table as LaTeX

This figure as: png pdf

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker