Publication

Lightning Pose: improved animal pose estimation (Nature Methods)

Science begets technology but technology also begets science. Anne Urai from the Leiden Institute of Psychology is part of an international team team that rolls out a new, user-friendly, plug-and-play AI tool they hope will become an engine of discovery by quantifying the growing volume of recorded animal behavior like never before. Now out in Nature Methods.

Author

Dan Biderman, Matthew R. Whiteway, Cole Hurwitz, Nicholas Greenspan, Robert S. Lee, Ankit Vishnubhotla, Richard Warren, Federico Pedraja, Dillon Noone, Michael M. Schartner, Julia M. Huntenburg, Anup Khanal, Guido T. Meijer, Jean-Paul Noel, Alejandro Pan-Vazquez, Karolina Z. Socha, Anne E. Urai, The International Brain Laboratory, John P. Cunningham, Nathaniel B. Sawtell & Liam Paninski

Date

25 June 2024

Links

Nature Methods: Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses.

Behavior is a window into the processes that underlie animal intelligence, ranging from early sensory processing to complex social interaction. Pose estimation methods based on fully supervised deep learning have emerged as a workhorse for behavioral quantification. This technology reduces high-dimensional videos of behaving animals to low-dimensional time series of their poses, defined in terms of a small number of user-selected keypoints per video frame. To improve the robustness and usability of animal pose estimation, we present Lightning Pose, a solution at three levels: modeling, software and a cloud-based application.