Events Calendar

PhD Dissertation Defense - Haiwei Chen

Title: Designing Neural Networks from the Perspective of Spatial Reasoning
 
Date: December 9, 2024
 
Time: 2:00 pm - 4:00 pm 
 
Location: GFS 104
 
Committee: Yajie Zhao (Chair), Ramakant Nevatia, and Andrew Nealen
 
Abstract: All visual data, from image to CAD models, live in a 2D or 3D spatial domain. In order to understand and model the visual data, spatial reasoning has always been fundamental to computer vision algorithms. Naturally, the practice has been widely extended to the use of artificial neural networks built for visual analysis. The basic building blocks of a neural network - operators and representations - are means to learn spatial relationships and therefore are built with spatial properties. In this thesis, we present the designs of ``spatial-aware'' neural operators and representations in different application contexts, with a unique focus on how these design choices affect the spatial properties of the neural networks in a way that is beneficial for the tasks at hand. The first topic explored is the equivariance property, where a SE(3) equivariant convolutional network is designed for 3D pose estimation and scene registration. In this chapter, we show that the equivariant property of a convolutional neural network can be practically extended to higher dimensional space and proved highly effective for applications that are not only sensitive to translation, but also 3D rotations. The second topic explored is learning neural operators that approximate spatially continuous function in a pattern synthesis application context. In this chapter, we explore novel representations of periodic encoding and a continuous latent space for a generative network that is able to synthesize diverse, high-quality and continuous 2D and 3D patterns. The unique formulation allows the generative model to be at least 10 times faster and more memory efficient compared to previous efforts, and marked one of the earliest attempts to adopt the implicit network to the generative setting. The third topic explored is spatial awareness with regard to incomplete images, where a generative network model for image inpainting is designed based on restricting its receptive field. Combined with the generative transformer and the discrete latent codes, this novel paradigm demonstrates the effectiveness of separating analysis and synthesis in challenging image inpainting scenarios, as the resulted network model achieves state-of-the-art performance in both diversity and quality, when completing partial images with free-form holes occupying as large as 70\% of the image. I believe that the topics covered have contributed to a better understanding of neural operator and representation designs for both discriminative and generative learning in computer vision, from a perspective of identifying the effective ways of spatial reasoning for the targeted visual applications. 

Location: Grace Ford Salvatori Hall Of Letters, Arts & Sciences (GFS) - 104

Audiences: Everyone Is Invited

Contact: Ellecia Williams