View a PDF of the paper titled Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Images, by Virmarie Maquiling and 4 other authors
Abstract:We explore the transformative potential of SAM 2, a vision foundation model, in advancing gaze estimation and eye tracking technologies. By significantly reducing annotation time, lowering technical barriers through its ease of deployment, and enhancing segmentation accuracy, SAM 2 addresses critical challenges faced by researchers and practitioners. Utilizing its zero-shot segmentation capabilities with minimal user input-a single click per video-we tested SAM 2 on over 14 million eye images from diverse datasets, including virtual reality setups and the world’s largest unified dataset recorded using wearable eye trackers. Remarkably, in pupil segmentation tasks, SAM 2 matches the performance of domain-specific models trained solely on eye images, achieving competitive mean Intersection over Union (mIoU) scores of up to 93% without fine-tuning. Additionally, we provide our code and segmentation masks for these widely used datasets to promote further research.
Submission history
From: Virmarie Maquiling [view email]
[v1]
Fri, 11 Oct 2024 15:50:53 UTC (1,825 KB)
[v2]
Thu, 12 Dec 2024 12:18:39 UTC (1,834 KB)
[v3]
Mon, 13 Jan 2025 15:19:14 UTC (1,834 KB)
Source link
lol