Department of Mathematics

Time and Place: Tuesdays 3:05-4:20 pm in Room 0231, Love Building

Course: MAP 6939

Scope: The Applied and Computational Math Seminar is series of talks with various topics covering a broad spectrum of not only applied and computational mathematics but also engineering. Researchers outside of the Department of Mathematics and Florida State University, postdocs and senior Ph.D students are also welcomed to share their work. Please contact the organizer if you wish to schedule your talk. For Spring 2023, contact Kyle Gallivan (gallivan"at"math.fsu.edu)

Abstract: Neural networks are usually trained with different variants of gradient descent-based optimization algorithms such as stochastic gradient descent or the Adam optimizer.  Recent theoretical work states that the critical points (where the gradient of the loss is zero) of two-layer ReLU networks with the square loss are not all local minima.  However, in this work we will explore an algorithm for training two-layer neural networks with ReLU-like activation and the square loss that alternatively finds the critical points of the loss function analytically for one layer while keeping the other layer and the neuron activation pattern fixed. Experiments on real and simulated data indicate that this simple algorithm can find deeper optima than the gradient descent-based algorithms when the data dimension is small, in a smaller number of iterations, and without any tuning parameters.

Bio: Adrian Barbu received a Ph.D. in Mathematics in 2000 from Ohio State University and a Ph.D. in Computer Science in 2005 from University of California, Los Angeles. From 2005 to 2007, he was a research scientist and later a project manager in Siemens Corporate Research, working on medical imaging problems. He received the 2011 Thomas A. Edison Patent Award with his Siemens coauthors for their work on Marginal Space Learning. In 2007, he joined the Statistics Department at Florida State University as an assistant professor and since 2019 as a professor. He has published more than 70 papers in computer vision, machine learning, and medical imaging and has more than 25 patents related to medical imaging and image denoising. He is the co-author, with his Ph.D. advisor Song-Chun Zhu, of the book "Monte Carlo Methods" published by Springer in 2020.