Events Calendar

PhD Thesis Defense - Hikaru Ibayashi

PhD Thesis Defense - Hikaru Ibayashi

Title:
Sharpness Analysis of Neural-networks for Physics Simulation

Committee members:
Prof. Aiichiro Nakano (Chair), Prof. Yan Liu, Prof. Paulo Branicio (Department of Chemical Engineering and Materials Science)

Abstract:
Deep learning has attracted significant attention in recent years due to its remarkable achievements in various applications. However, building effective deep neural networks requires making crucial design choices such as the network architecture, regularization, optimization, and hyperparameter tuning.
In this dissertation, we focus on the concept of ``sharpness'' of neural networks,
which refers to neural networks' sensitivity against perturbation on weight parameters. We argue that sharpness is not only a theoretical notion but also has practical use cases that can lead to better generalization and robustness of neural models.

A major theoretical challenge of defining and measuring sharpness is its scale-sensitivity, i.e., the fact that sharpness can change to the scale transformation of neural networks. In this thesis, we propose a novel definition of sharpness that overcomes this limitation, with provable scale-invariance and extensive empirical validation. By analyzing the relationship between sharpness and model performance, We show how my definition can provide a more objective and accurate characterization of sharpness.

Another open question in the sharpness analysis is how training algorithms for machine learning models regularize sharpness. In this dissertation, we answer this question by showing that existing training algorithm methods regularize sharpness through what can be called "escaping" behavior, where the optimization process avoids sharp regions in the parameter space. This new explanation demystifies the connection between sharpness and training algorithms, paving the way for more effective and principled approaches to machine learning.

Finally, we demonstrate the practical benefits of sharpness regularization for physics simulations. We show that neural networks with small sharpness achieve high-fidelity fluid simulation and molecular dynamics. These findings include the significant implication that sharpness is not just a mathematical notion but also a practical tool for building physics-informed neural networks.

Location: Seaver Science Library (SSL) - 104

Audiences: Everyone Is Invited

Contact: Melissa Ochoa