Select a calendar:
Filter May Events by Event Type:
SUNMONTUEWEDTHUFRISAT
University Calendar
Events for May
-
PhD Thesis Proposal - Ta-Yang Wang
Wed, May 01, 2024 @ 03:00 PM - 04:00 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Training Heterogeneous Graph Neural Networks using Bandit Sampling
Presenter: Ta-Yang Wang
Time: May 1st, 3:00 PM - 4:00 PM
Location: EEB 219
Committee members: Viktor Prasanna (chair), Jyotirmoy Deshmukh, Rajgopal Kannan, Aiichiro Nakano, and Cauligi Raghavendra
Abstract: Graph neural networks (GNNs) have gained significant attention across diverse areas due to their superior performance in learning graph representations. While GNNs exhibit superior performance compared to other methods, they are primarily designed for homogeneous graphs, where all nodes and edges are of the same type. Training a GNN model for large-scale graphs incurs high computation and storage costs, especially when considering the heterogeneous structural information of each node. To address the demand for efficient GNN training, various sampling methods have been proposed. In this proposal, we hypothesize that one can improve the training efficiency via bandit sampling, an online learning algorithm with provable convergence under weak assumptions on the learning objective. The main idea is to prioritize node types with more informative connections with respect to the learning objective. Additionally, we analyze the limitations of the framework, thus advancing its applicability in large-scale graph learning tasks.Location: Hughes Aircraft Electrical Engineering Center (EEB) - 219
Audiences: Everyone Is Invited
Contact: Ellecia Williams
-
PhD Thesis Defense - Matthew Ferland
Thu, May 02, 2024 @ 12:00 PM - 02:00 PM
Thomas Lord Department of Computer Science
University Calendar
PhD Thesis Defense: Matthew Ferland
Committee: Shanghua Teng (Chair), David Kempe, Jiapeng Zhang, Larry Goldstein (Math)
Title: Exploring the Computational Frontier of Combinatorial Games
Abstract: People have been playing games since before written history, and many of the earliest games were combinatorial games, that is to say, games of perfect information and no chance. This type of game is still widely played today, and many popular games of this type, such as Chess and Go, are some of the most studied games of all time. This proposed work resolves around a game-independent systemic study of these games. More specifically, computational properties involving evaluating mathematical analysis tools for combinatorial games, such as Grundy values and confusion intervals, as well as identifying what can be determined about these games using simple oracle models.Location: Henry Salvatori Computer Science Center (SAL) - 213
Audiences: Everyone Is Invited
Contact: CS Events
-
PhD Dissertation Defense - I-Hung Hsu
Tue, May 07, 2024 @ 02:10 PM - 04:00 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Towards Generalized Event Understanding in Text via Generative Models
Committee Members: Dr. Prem Natarajan (Chair), Dr. Nanyun Peng (Co-Chair), Dr. Dan O'Leary, Dr. Emilio Ferrara
Date and Time: May 7th, 2024 - 2:10p - 4:00p
Abstract: Human languages in the world, such as news or narratives, are structured around events. Focusing on these events allows Natural Language Processing (NLP) systems to better understand plots, infer motivations, consequences, and the dynamics of situations. Despite the rapidly evolving landscape of NLP technology, comprehending complex events, particularly those rarely encountered in training such as in niche domains or low-resource languages, remains a formidable challenge. This thesis explores methods to enhance NLP model generalizability for better adaptability to unfamiliar events and languages unseen during training.
My approach includes two main strategies: (1) Model Perspective: I propose a novel generation-based event extraction framework, largely different from typical solutions that make predictions by learning to classify input tokens. This new framework utilizes indirect supervision from natural language generation, leveraging large-scale unsupervised data without requiring additional training modules dependent on limited event-specific data. Hence, it facilitates the models’ ability on understanding general event concepts. I further explore advanced methods to extend this framework for cross-lingual adaptation and to utilize cross-domain robust resources effectively. (2) Data Perspective: I develop techniques to generate pseudo-training data broaden the training scope for event understanding models. This includes translating structured event labels into other languages with higher accuracy and fidelity, and synthesizing novel events for the existing knowledge base.
Overall, my work introduces a novel learning platform to the NLP community, emphasizing an innovative modeling paradigm and comprehensive data preparation to foster more generalized event understanding models.
Location: Information Science Institute (ISI) - 727
Audiences: Everyone Is Invited
Contact: I-Hung Hsu
Event Link: https:/usc.zoom.us/j/95785927723?pwd=dFlGbEcwbXlGalJ6OVk3YW41RDMrdz09
-
PhD Thesis Defense - Qinyi Luo
Thu, May 09, 2024 @ 11:00 AM - 02:00 PM
Thomas Lord Department of Computer Science
University Calendar
PhD Thesis Defense - Qinyi (Chelsea) Luo
Committee members: Xuehai Qian (co-chair), Viktor Prasanna (co-chair), Ramesh Govindan, Chao Wang, Feng Qian
Title: High-Performance Heterogeneity-Aware Distributed Machine Learning Model Training
Abstract: The increasing size of machine learning models and the ever-growing amount of data result in days or even weeks of time required to train a machine learning model. To accelerate training, distributed training with parallel stochastic gradient descent is widely adopted as the go-to training method. This thesis targets four challenges in distributed training: (1) performance degradation caused by large amount of data transfer among parallel workers, (2) heterogeneous computation and communication capacities in the training devices, i.e., the straggler issue, (3) huge memory consumption during training caused by gigantic model sizes, and (4) automatic selection of parallelization strategies. This thesis first delves into the topic of decentralized training and proposes system support and algorithmic innovation that strengthen tolerance against stragglers in data-parallel training. On the system side, a unique characteristic of decentralized training, the iteration gap, is identified, and a queue-based synchronization mechanism is proposed to efficiently support decentralized training as well as common straggler-mitigation techniques. In the experiments, the proposed training protocol, Hop, can provide strong tolerance against stragglers and train much faster than standard decentralized training when stragglers are present. On the algorithm side, a novel communication primitive, randomized partial All-Reduce, is proposed to enable fast synchronization in decentralized data-parallel training. The proposed approach, Prague, can achieve a 1.2x speedup against All-Reduce in a straggler-free environment and a 4.4x speedup when stragglers are present. Then, on the topic of memory optimization for training Deep Neural Networks (DNNs), an adaptive during-training model compression technique, FIITED, is proposed to reduce the memory consumption of training huge recommender models. FIITED adapts to dynamic changes in data and adjusts the dimension of each individual embedding vector continuously during training. Experiments show that FIITED is able to reduce the memory consumption of training significantly more than other embedding pruning methods, while maintaining the trained model's quality. In the end, in the aspect of automatic parallelization of training workloads, a novel unified representation of parallelization strategies, incorporating Data Parallelism (DP), Model Parallelism (MP) and Pipeline Parallelism (PP), is proposed, as well as a search algorithm that selects superior parallel settings in the vast search space. An ideal stage partition ratio for synchronous pipelines is derived for the first time, to the best of my knowledge, and it is theoretically proven that unbalanced partitions are better than balanced partitions. In addition, by examining the pipeline schedule, a trade-off between memory and performance is uncovered and explored. Experiments show that hybrid parallel strategies generated with the aforementioned optimizations consistently outperform those without such considerations.
Date: May 9, 2024
Time: 11:00 a.m. - 1:00 p.m.
Location: EEB 110
Zoom link: https://usc.zoom.us/j/95741130954?pwd=dkRkblNlNGt0TlkwOU51SlRNS0hPZz09Location: Hughes Aircraft Electrical Engineering Center (EEB) -
Audiences: Everyone Is Invited
Contact: CS Events
Event Link: https://usc.zoom.us/j/95741130954?pwd=dkRkblNlNGt0TlkwOU51SlRNS0hPZz09
-
PhD Dissertation Defense - Binh Vu
Fri, May 17, 2024 @ 03:00 PM - 05:00 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Exploiting Web Tables and Knowledge Graphs for Creating Semantic Descriptions of Data Sources
Committee: Craig Knoblock (Chair), Sven Koenig, Daniel Edmund O'Leary, Yolanda Gil, Jay Pujara
Date and Time: Friday, May 17th - 3:00p - 5:00p
Location: SAL 322
Abstract: There is an enormous number of tables available on the web, and they can provide valuable information for diverse applications. To harvest information from the tables, we need precise mappings, called semantic descriptions, of concepts and relationships in the data to classes and properties in a target ontology. However, creating semantic descriptions, or semantic modeling, is a complex task requiring considerable manual effort and expertise. Much research has focused on automating this problem. However, existing supervised and unsupervised approaches both face various difficulties. The supervised approaches require lots of known semantic descriptions for training and, thus, are hard to apply to a new or large domain ontology. On the other hand, the unsupervised approaches exploit the overlapping data between tables and knowledge graphs; hence, they perform poorly on tables with lots of ambiguity or little overlapping data. To address the aforementioned weaknesses, we present novel approaches for two main cases: tables that have overlapping data with a knowledge graph (KG) and tables that do not have overlapping data. Exploiting web tables that have links to entities in a KG, we automatically create a labeled dataset to learn to combine table data, metadata, and overlapping background knowledge (if available) to find accurate semantic descriptions. Our methods for the two cases together provide a comprehensive solution to the semantic modeling problem. In the evaluation, our approach in the overlapping setting yields an improvement of approximately 5\% in F$_1$ scores compared to the state-of-the-art methods. In the non-overlapping setting, our approach outperforms strong baselines by 10\% to 30\% in F$_1$ scores.Location: Henry Salvatori Computer Science Center (SAL) - 322
Audiences: Everyone Is Invited
Contact: Felante' Charlemagne
-
PhD Dissertation Defense - Avi Thawani
Tue, May 21, 2024 @ 01:30 PM - 03:30 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Aggregating Symbols fo Language Modeling
Date and Time: Tuesday, May 21st, 2024 - 1:30p - 3:30p
Committee: Jay Pujara (Chair), Swabha Swayamdipta, Dani Yogatama, Aiichiro Nakano, Gerard Hoberg
Abstract: Natural language is a sequence of symbols. Language Models (LMs) are powerful at learning sequence patterns. The first step for large language models (LLMs) like ChatGPT is to convert text (that humans understand) into indices (that models do). This crucial phase in the Language Modeling pipeline has unfortunately been understudied and is currently achieved by subword segmentation, a manually engineered set of heuristics. I will deep dive into case studies where these heuristics fail and my recommended improvements: for example when representing numbers in text, as well as multi-word phrases. I present an end-to-end tokenized language model that understands both words and numbers better than subwords without any manually engineered heuristic. It also outperforms character-level tokenisation, promising up to 4/6x speed up in inference and training respectively.
I show the benefits of aggregating symbols for language modeling, and investigate key aspects of symbol use in LMs:
1. Aggregating on the number line improves both numeracy and literacy of language models
2. We can learn to aggregate symbols given a corpus with improved language modeling and approximate
3. Learning to aggregate symbols helps downstream performance in certain application areas like neural machine translation of non-concatenative languages
Zoom Link: https://usc.zoom.us/j/96005480765?pwd=TXFUWU5KWjA1S3JtM3FNaWRQZVZOZz09Location: Hughes Aircraft Electrical Engineering Center (EEB) - 110
Audiences: Everyone Is Invited
Contact: Felante' Charlemagne
