Select a calendar:
Filter February Events by Event Type:
SUNMONTUEWEDTHUFRISAT
Events for February 19, 2019
-
CS Colloquium: Jason Lee (USC, Data Sciences and Operations)On the Foundations of Deep Learning: SGD, Overparametrization, and Generalization
Tue, Feb 19, 2019 @ 11:00 AM - 12:00 PM
Thomas Lord Department of Computer Science
Conferences, Lectures, & Seminars
Speaker: Jason Lee, USC, Data Sciences and Operations
Talk Title: On the Foundations of Deep Learning: SGD, Overparametrization, and Generalization
Series: CS Colloquium
Abstract: We provide new results on the effectiveness of SGD and overparametrization in deep learning.
a) SGD: We show that SGD converges to stationary points for general nonsmooth , nonconvex functions, and that stochastic subgradients can be efficiently computed via Automatic Differentiation. For smooth functions, we show that gradient descent, coordinate descent, ADMM, and many other algorithms, avoid saddle points and converge to local minimizers. For a large family of problems including matrix completion and shallow ReLU networks, this guarantees that gradient descent converges to a global minimum.
b) Overparametrization: We show that gradient descent finds global minimizers of the training loss of overparametrized deep networks in polynomial time.
c) Generalization:
For general neural networks, we establish a margin-based theory. The minimizer of the cross-entropy loss with weak regularization is a max-margin predictor, and enjoys stronger generalization guarantees as the amount of overparametrization increases.
d) Algorithmic and Implicit Regularization: We analyze the implicit regularization effects of various optimization algorithms on overparametrized networks. In particular we prove that for least squares with mirror descent, the algorithm converges to the closest solution in terms of the bregman divergence. For linearly separable classification problems, we prove that the steepest descent with respect to a norm solves SVM with respect to the same norm. For over-parametrized non-convex problems such as matrix sensing or neural net with quadratic activation, we prove that gradient descent converges to the minimum nuclear norm solution, which allows for both meaningful optimization and generalization guarantees
This lecture satisfies requirements for CSCI 591: Research Colloquium
Biography: Jason Lee is an assistant professor in Data Sciences and Operations at the University of Southern California. Prior to that, he was a postdoctoral researcher at UC Berkeley working with Michael Jordan. Jason received his PhD at Stanford University advised by Trevor Hastie and Jonathan Taylor. His research interests are in statistics, machine learning, and optimization. Lately, he has worked on high dimensional statistical inference, analysis of non-convex optimization algorithms, and theory for deep learning.
Host: Yan Liu
Location: Olin Hall of Engineering (OHE) - 132
Audiences: Everyone Is Invited
Contact: Assistant to CS chair
-
PhD Defense - Katelyn Swift-Spong
Tue, Feb 19, 2019 @ 01:30 PM - 03:30 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Towards Socially Assistive Robot Support Methods for Physical Activity Behavior Change
PhD Candidate: Katelyn Swift-Spong
Date, Time, and Location: Tuesday, February 19, 2019 at 1:30pm in RTH 406
Committee: Prof. Maja MatariÄ (chair), Prof. Stefanos Nikolaidis, and Prof. Elizabeth Zelinski
Abstract:
Socially Assistive Robot (SAR) systems have the potential to support the complex process of human behavior change by providing social support such as feedback and encouragement at opportune times. This dissertation presents a framework for SAR behavior change support in the context of physical activity behavior. This framework is designed around the goal of creating lasting behavior change that extends past the SAR interaction. Within this framework, the robot is equipped with one or more SAR physical activity behavior change support methods designed to affect a specific mechanism of behavior change.
This dissertation develops the design of SAR feedback, backstory, and messaging support methods for physical activity behavior change. These three methods were each designed to support a different mechanism of achieving behavior change by leveraging the robot's relational and support capabilities. Feedback was designed to support a user's beliefs about their ability to perform a physical activity task. Robot backstory was designed to increase the robot's ability to provide social support, and messaging was designed to increase the user's positive feelings towards the physical activity. These three support methods are evaluated in real-world physical activity domains with a fully autonomous SAR system. The feedback support method is evaluated in the domain of post-stroke rehabilitation, and the backstory and messaging support methods are evaluated in the domain of adolescent exercise.
Reminder and social reward decision making is also developed as a SAR physical activity behavior change support method using a model of SAR habit formation support. This model formalizes the SAR sequential decision making task of determining when to give reminders and social rewards towards the goal of supporting the formation of a new desired habit. Habits are formed when the occurrence of a cue is followed by a desired behavior, and that combination is reinforced repeatedly over time. The model of habit formation support enables a robot to intervene in this process. This model is evaluated in the domain of reducing older adult sedentary behavior through a two-week in-home SAR intervention. The robot was able to generate a high level of reminder adherence in this setting.
In this work, four SAR physical activity behavior change support methods were developed and evaluated in three different physical activity domains with fully autonomous SAR systems. This dissertation contributes to understanding the methods a robot could use to support behavior change in a variety of physical activity domains both in situ within the context of the behavior in everyday life and outside of that context.
Location: Ronald Tutor Hall of Engineering (RTH) - 406
Audiences: Everyone Is Invited
Contact: Lizsl De Leon
