Lecture
SAILS Lunch Time Seminar: Rob van Nieuwpoort
- Date
- Monday 23 September 2024
- Time
- Location
- Online only
Training Larger AI Models
Deep learning has dramatically improved the state-of-the-art in object detection, speech recognition, anomaly detection, natural language processing, and many other domains. However, model sizes keep increasing quickly, and we need more and more memory and computing resources to train our models.
In this talk, we will discuss methods to make model training more efficient, as well as ways to scale up, utilising larger compute resources effectively. We will describe the various local, national and international compute resources that are available to SAILS researchers. Next, we will explain methods to train models exploiting different types of parallelism, such as data, model, tensor, pipeline, and hybrid parallelism. Finally, we will introduce a novel method that reduces the memory usage when training on multiple GPUs in parallel.
Join us!
The SAILS Lunch Time Seminar is an online event, but it is not publicly accessible in real-time. Please click the the link below to register to our mailinglist and receive participation links for our Lunch Time Seminars.
Click here to register