Logo: University of Southern California

Events Calendar

  • PhD Defense Dong Guo

    Tue, Jul 17, 2018 @ 11:00 AM - 01:00 PM

    Computer Science

    University Calendar

    PhD Candidate: Dong Guo

    Title: Learning useful and interpretable representations via information regularizations

    Raghu Raghavendra (chair), Aiichiro Nakano, Viktor K Prasanna (outside member)

    EEB 248

    July 17th



    We studied the application of the Information Bottleneck (IB) principle in two machine learning problems. The IB principle suggests learning representations that is maximally relevant to predictions while being maximally compressive about input data, and it is considered as one possible way of explaining the black box in successful deep learning algorithm.

    The first application is analyzing and designing supervised classifier, focusing on the relevance between representation and predictions. We used to observe that entropy regularized log-likelihood (ERLL) was a good model selection criterion when we trained acoustic state classifier for acoustic speech recognition. Starting from IB principle, we derived an approximate lower bound of IB objective that can explain the strength of ERLL in model selection, and accordingly proposed heuristic algorithm that uses entropy to learn classifiers that are both accurate and confident. We demonstrate it on multiple benchmark datasets.

    The second application is unsupervised learning of interpretable representation, focusing on the compression of input data. We proposed a variant of variational autoencoder (VAE) model that jointly learn one representation that encodes absract concepts and one representation that encodes details of input data. This model architecture provides a flexible way of balancing the task of informative features extraction by encoders and samples generation by decoders. We demonstrate it on application of clustering analysis and concept discovery in representation space.

    Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

    Audiences: Everyone Is Invited

    Posted By: Lizsl De Leon


Return to Calendar