Logo: University of Southern California

Events Calendar



Select a calendar:



Filter February Events by Event Type:

<< Previous Month
Next Month >>

SUNMONTUEWEDTHUFRISAT
28
29
30
31
(2)
3

4
6
8
9
10

11
12
15
16
17

18
19
21
(1)
23
24

25
26
27
29
1
2


Events for February 21, 2024

  • Feb

    21

    PhD Thesis Proposal - Qinyi Luo

    Wed, Feb 21, 2024 @ 11:00 AM - 12:30 PM

    Thomas Lord Department of Computer Science

    University Calendar


    PhD Thesis Proposal - Qinyi Luo
    Title: High-Performance Heterogeneity-Aware Distributed Machine Learning Model Training
     
    Committee members: Xuehai Qian (co-chair), Viktor Prasanna (co-chair), Ramesh Govindan, Chao Wang, Salman Avestimehr    
     
    Abstract: The increasing size of machine learning models and the ever-growing amount of data result in days or even weeks of time required to train a machine learning model. To accelerate training, distributed training with parallel stochastic gradient descent is widely adopted as the go-to training method. This thesis proposal targets four challenges in distributed training: (1) performance degradation caused by large amount of data transfer among parallel workers, (2) heterogeneous computation and communication capacities in the training devices, i.e., the straggler problem, (3) huge memory consumption during training caused by huge model sizes, and (4) automatic selection of parallelization strategies. The proposal first introduces our work in decentralized training, including system support and algorithmic innovation that strengthen tolerance against stragglers in data-parallel training. Then, an adaptive during-training model compression technique is proposed to reduce the memory consumption of training huge recommender models. In the end, in the aspect of automatic parallelization of training workloads, a novel unified representation of parallelization strategies is proposed, as well as a search algorithm that selects superior parallel settings in the vast search space, and preliminary findings are discussed.     
     
    Date and time: Feb 21 11am-12:30pm
    Location: EEB 110  
     
    Zoom link: https://usc.zoom.us/j/97299158202?pwd=bVlnRVFhTjJlZjVCY1hVNy9yWWE1UT09          

    Location: Hughes Aircraft Electrical Engineering Center (EEB) - 110

    Audiences: Everyone Is Invited

    Contact: CS Events

    OutlookiCal