Events Calendar

PhD Defense - Zhiyun Lu

Ph.D. Candidate: Zhiyun Lu
Date: Wednesday, May 20, 2020
Time: 12:00 PM - 2:00 PM
Committee: Fei Sha (Chair), Haipeng Luo, C.-C. Jay Kuo

Title: Leveraging Training Information for Efficient and Robust Deep Learning

Abstract: Deep neural nets have exhibited great success on a wide range of machine learning problems across various domains, such as speech, image, and text. Despite decent prediction performances, there are rising concerns for the `in-the-lab' machine learning models to be vastly deployed in the wild. In this thesis, we study two of the main challenges in deep learning: efficiency, computational as well as statistical, and robustness. We describe a set of techniques to solve the challenges by utilizing information from the training process intelligently. The solutions go beyond the common recipe of a single point estimate of the optimal model.

The first part of the thesis studies the efficiency challenge. We propose a budgeted hyper-parameter tuning algorithm to improve the computation efficiency of hyper-parameter tuning in deep learning. It can estimate and utilize the trend of training curves to adaptively allocate resources for tuning, which demonstrates improved efficiency over state-of-the-art tuning algorithms. Then we study the statistical efficiency on tasks with limited labeled data. Specifically we focus on the task of speech sentiment analysis. We apply pre-training using automatic speech recognition data, and solve sentiment analysis as a downstream task, which greatly improves the data efficiency of sentiment labels.



The second part of the thesis studies the robustness challenge. Motivated by the resampling method in statistics, we study the uncertainty estimate of neural networks by local perturbative approximations. We propose to sample replicas of the model parameters from a Gaussian distribution to form a pseudo-ensemble. The ensemble predictions are used to estimate the uncertainty of the original model, which improves its robustness against invalid inputs.


Meeting links:


Zoom: https://usc.zoom.us/j/96089712182 (Meeting ID: 960 8971 2182)


Google Meet (backup): meet.google.com/nxz-eybf-urw

WebCast Link: https://usc.zoom.us/j/96089712182

Audiences: Everyone Is Invited

Contact: Lizsl De Leon