Events Calendar

PhD Defense Dong Guo

PhD Candidate: Dong Guo

Title: Learning useful and interpretable representations via information regularizations

Raghu Raghavendra (chair), Aiichiro Nakano, Viktor K Prasanna (outside member)

EEB 248

July 17th

11am~1pm

Abstract:

We studied the application of the Information Bottleneck (IB) principle in two machine learning problems. The IB principle suggests learning representations that is maximally relevant to predictions while being maximally compressive about input data, and it is considered as one possible way of explaining the black box in successful deep learning algorithm.

The first application is analyzing and designing supervised classifier, focusing on the relevance between representation and predictions. We used to observe that entropy regularized log-likelihood (ERLL) was a good model selection criterion when we trained acoustic state classifier for acoustic speech recognition. Starting from IB principle, we derived an approximate lower bound of IB objective that can explain the strength of ERLL in model selection, and accordingly proposed heuristic algorithm that uses entropy to learn classifiers that are both accurate and confident. We demonstrate it on multiple benchmark datasets.

The second application is unsupervised learning of interpretable representation, focusing on the compression of input data. We proposed a variant of variational autoencoder (VAE) model that jointly learn one representation that encodes absract concepts and one representation that encodes details of input data. This model architecture provides a flexible way of balancing the task of informative features extraction by encoders and samples generation by decoders. We demonstrate it on application of clustering analysis and concept discovery in representation space.

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

Audiences: Everyone Is Invited

Contact: Lizsl De Leon