Events Calendar

CS Colloquium Lecture Series: Rich Caruana (Microsoft Research) - Do Deep Nets Really Need To Be Deep?

Speaker: Rich Caruana , Microsoft Research

Talk Title: Do Deep Nets Really Need To Be Deep?

Series: CS Colloquium

Abstract: Currently, deep neural networks are the state of the art on problems such as speech recognition and computer vision. By using a method called model compression, we show that shallow feed-forward nets can learn the complex functions previously learned by deep nets and achieve accuracies previously only achievable with deep models while using the same number of parameters as the original deep models. On the TIMIT phoneme recognition and CIFAR-10 image recognition tasks, shallow nets can be trained that perform similarly to complex, well-engineered, deeper convolutional architectures. The same model compression trick can also be used to compress impractically large deep models and ensembles of large deep models down to “medium-size” deep models that run more efficiently on servers, and down to “small” models that can run on mobile devices. In machine learning and statistics we used to believe that one of the keys to preventing overfitting was to keep models simple and the number of parameters small to force generalization. We no longer believe this --- learning appears to generalize best when training models with excess capacity, but the learned functions can often be represented with far fewer parameters.

The lecture can be streamed HERE

Host: Yan Liu

Location: Henry Salvatori Computer Science Center (SAL) - 101

Audiences: Everyone Is Invited

Contact: Assistant to CS chair