Past Event: Babuška Forum

Side-effects of learning from low dimensional data embedded in a Euclidean space

Yen-Hsi Richard Tsai, Professor, Department of Mathematics, Oden Institute, UT Austin

Abstract

The low dimensional manifold hypothesis posits that the data found in many applications, such as those involving natural images, lie (approximately) on low dimensional manifolds embedded in a high dimensional Euclidean space. In this setting, a typical neural network defines a function that takes a finite number of vectors in the embedding space as input. However, one often needs to consider evaluating the optimized network at points outside the training distribution. We analyze the cases where the training data are distributed in a linear subspace of Rd. We derive estimates on the variation of the learning function, defined by a neural network, in the direction transversal to the subspace. We study the potential regularization effects associated with the network’s depth and noise in the codimension of the data manifold. 

This is a joint work with Juncai He and Rachel Ward.

Biography

Richard Tsai received his PhD in Mathematics in 2002 from UCLA. He was a Veblen Instructor at Princeton University and the Institute for Advanced Study before joining UT. Richard was born in Taiwan and has lived in the United States since 1997.

Richard's current focus is on developing machine learning approaches that complement the more classical techniques for challenging scientific computing tasks. The types of scientific computing problems are multiscale coupling algorithms for initial value problems, nonlinear interface dynamics, partial differential equations on surfaces, multiscale modeling, wave propagation, image processing, sensor networks, and robotic path planning problems.