BasketEvent: Understanding Who Did What and When in Basketball Videos

The paper introduces a novel player-centric approach to basketball video understanding, focusing on identifying "who did what and when" in broadcast clips. This approach shifts the focus from video-level to player-event instances, aiming to ground event labels to responsible players and localize the temporal intervals of decisive evidence.

To support this task, the authors developed BasketEvent, a dataset curated from real NBA broadcasts. Using the NBA API, they collected play-by-play records and corresponding event-centered clips, transforming video-level metadata into player-level event annotations. This dataset includes manually annotated samples for evaluating temporal localization.

The authors propose PlayNet, a reasoning framework that processes broadcast videos to extract player and ball trajectories, associates them with identities, and models interactions to predict player-level events and their temporal intervals. PlayNet uses trajectory-guided visual tokens and global court context to enhance its predictions.

Experiments demonstrate that PlayNet outperforms traditional video-level and crop-based baselines, providing more detailed and interpretable basketball video understanding. The framework's ability to ground events in both players and time allows for fine-grained and temporally precise analysis.

The study acknowledges limitations in handling spatially close players and brief event evidence, suggesting that player-centric basketball understanding remains challenging. Future work could explore more sophisticated interaction modeling and temporal evidence localization to address these challenges.

Overall, the paper contributes a new task formulation, a comprehensive dataset, and a robust framework for structured basketball video understanding, laying the groundwork for further advancements in the field.