BankTweak: Adversarial Attack against Multi-Object Trackers by Manipulating Feature Banks

arXiv:2408.12727v1 Announce Type: new
Abstract: Multi-object tracking (MOT) aims to construct moving trajectories for objects, and modern multi-object trackers mainly utilize the tracking-by-detection methodology. Initial approaches to MOT attacks primarily aimed to degrade the detection quality of the frames under attack, thereby reducing accuracy only in those specific frames, highlighting a lack of textit{efficiency}. To improve efficiency, recent advancements manipulate object positions to cause persistent identity (ID) switches during the association phase, even after the attack ends within a few frames. However, these position-manipulating attacks have inherent limitations, as they can be easily counteracted by adjusting distance-related parameters in the association phase, revealing a lack of textit{robustness}. In this paper, we present textsf{BankTweak}, a novel adversarial attack designed for MOT trackers, which features efficiency and robustness. textsf{BankTweak} focuses on the feature extractor in the association phase and reveals vulnerability in the Hungarian matching method used by feature-based MOT systems. Exploiting the vulnerability, textsf{BankTweak} induces persistent ID switches (addressing textit{efficiency}) even after the attack ends by strategically injecting altered features into the feature banks without modifying object positions (addressing textit{robustness}). To demonstrate the applicability, we apply textsf{BankTweak} to three multi-object trackers (DeepSORT, StrongSORT, and MOTDT) with one-stage, two-stage, anchor-free, and transformer detectors. Extensive experiments on the MOT17 and MOT20 datasets show that our method substantially surpasses existing attacks, exposing the vulnerability of the tracking-by-detection framework to textsf{BankTweak}.



Source link
lol