Publication | Nature Methods

Lightning Pose: improved animal pose estimation

Science begets technology but technology also begets science. Anne Urai from the Leiden Institute of Psychology is part of an international team team that rolls out a new, user-friendly, plug-and-play AI tool: LightningPose, a tool for easy-to-use video tracking of animal behavior. They hope this will become an engine of discovery by quantifying the growing volume of recorded animal behavior. Now out in Nature Methods.

Author

Dan Biderman, Anne E. Urai et al.

Date

25 June 2024

Links

Nature Methods: Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses.

Behavior is a window into the processes that underlie animal intelligence, ranging from early sensory processing to complex social interaction. Pose estimation methods based on fully supervised deep learning have emerged as a workhorse for behavioral quantification. This technology reduces high-dimensional videos of behaving animals to low-dimensional time series of their poses, defined in terms of a small number of user-selected keypoints per video frame. To improve the robustness and usability of animal pose estimation, we present Lightning Pose, a solution at three levels: modeling, software and a cloud-based application.

All authors: Dan Biderman, Matthew R. Whiteway, Cole Hurwitz, Nicholas Greenspan, Robert S. Lee, Ankit Vishnubhotla, Richard Warren, Federico Pedraja, Dillon Noone, Michael M. Schartner, Julia M. Huntenburg, Anup Khanal, Guido T. Meijer, Jean-Paul Noel, Alejandro Pan-Vazquez, Karolina Z. Socha, Anne E. Urai, The International Brain Laboratory, John P. Cunningham, Nathaniel B. Sawtell & Liam Paninski