Michael Lew explores how computers can see
Learning how computers can process and describe images just like human beings do. It is one of the key elements of the research of Michael Lew, who sees Deep Learning as a promising way to achieve this goal. On the 1st of January 2021, he was appointed Professor of Deep Learning at the Leiden Institute for Advanced Computer Science (LIACS).
'Bridging' the semantic gap
One of Lew’s research focusses is on multimedia retrieval, which concentrates on the way human beings find digital media, such as images. To this end, it is essential that computers learn to ‘see’. ‘The grand challenge in multimedia retrieval has been ‘bridging’ the semantic gap. This is the gap between the high level concepts of humans and the low level features from images. The problem is that computers cannot yet describe what images show in the way humans can. If we can solve this problem and bridge the semantic gap, it would make all of the collected art and heritage, as well as scientific, internet and personal images searchable and accessible,’ says Lew.
Deep Learning
To achieve this goal researchers can use Deep Learning. ‘Deep Learning is currently one of the most impactful and promising areas of Artificial Intelligence,’ Lew explains. ‘It is a way to analyze large sets of data and extract the relevant data. The main difference with other forms of Artificial Intelligence is that it is modelled after the way in which the neural intelligence of the human brain works. Instead of a shallow approach, Deep Learning uses a hierarchy of many layers to process data. Each layer carries out its own tasks, making the data-analysis more detailed the deeper you go. Hence, these complex computational models use millions of neurons and can process enormous sets of data.’
Developing new neural networks
Lew mainly works on the development of new neural networks and architectures for multimedia, such as images, text, video and audio. Lew: ‘These new models give insights in three aspects of the neural architectures: how we can teach the computer brain new ideas, why it makes certain decisions and how we can increase its accuracy. Furthermore, they will allow the neural networks to gain new abilities, such as continuously learning new concepts.’
From 2D to 3D images
One of Lew’s current projects involves high dimensional deep learning. ‘In the recent past, most of the focus of deep learning models was on 2D image data,’ Lew clarifies. ‘However, 3D images are getting more and more important, for example in 3D MRI scans, automatic automobile driving and simulation data that are used for airflow and engine design. In this project, the focus is on developing new neural models which are specifically designed at these kinds of high dimensional applications.’
Teaching humans and computers
Lew has three specific goals he aims to be working on during his professorship. ‘My first goal is to endow computer neural networks with the ability to continuously learn new concepts using approaches inspired by the human brain. Currently, most neural networks are designed for a fixed number of visual concepts. For many situations however, that is highly unrealistic. Within the DeepMark Project, we are working on ways to improve this. Secondly, I want to create a better and broader understanding of neural networks. They are now perceived to be difficult to understand. I want to develop interactive visualizations of the neural networks and activations to create more insight in the way they function. Thirdly, there is a major need for very large training sets for real world contexts. My research group and our collaborators have already been highly active in releasing many of the top datasets in the world and we expect to continue doing this.’
Text: Chris Flinterman