CVPR 2023 - Vancouver, Canada

MammalNet: A Large-scale Video Benchmark for Mammal Recognition and Behavior Understanding

Arxiv Code


MammalNet Video Dataset


We propose MammalNet, a large-scale video benchmark for recognizing mammals and their behavior. It is built around a biological mammal taxonomy spanning 17 orders, 69 families and 173 mammal categories, and includes 12 common high-level mammal behaviors (e.g., hunt, groom).
MammalNet enables the study of animal and behavior recognition, both separately and jointly. It also facilitates investigating challenging compositional scenarios which test models' zero- and low-shot transfer abilities. Moreover, MammalNet includes behavior detection by localizing when a behavior occurs in an untrimmed video. Our dataset is the first to enable animal behavior analysis at scale in an ecologically-grounded manner, and exemplifies multiple challenges for the computer vision community, such as recognition of imbalanced, hierarchical distributions of fine-grained categories and generalization to unseen or seldom seen scenarios.


The MammamNet dataset for both challenge tracks will be made available soon.


If you encounter any technical issue related to the dataset, or if you're missing critical information, please open a ticket on our GitHub repository.


We only provide the annotations and do not distribute the videos. The licenses for our annotations are as follows: CC BY license.


Dataset Organizers

Jun Chen

Ph.D. Student KAUST

Ming Hu

Ph.D. Student Monash University

Darren J. Cooker

Research Scientist KAUST

Michael L. Berumen

Professor KAUST

Blair Costelloe

Behavioral Ecologist Max Planck Institute of Animal Behavior

Sara Beery

Assistant Professor MIT

Anna Rohrbach

Research Scientist University of California, Berkeley

Mohamed Elhoseiny

Assistant Professor KAUST