Invited talk @ Center for Statistical Science, Tsinghua University

Created in July 07, 2023

2023

Thanks Dr. Qian Lin for inviting me!

The title of my talk is Towards A Statistical Understanding of Neural Network Classifiers, which incorporates 3 of my recent works.

Abstract: Deep learning has made remarkable empirical strides in large-scale classification problems, yet a comprehensive statistical understanding remains elusive. Viewing classification as an estimation problem, there are two primary targets: the conditional probability and the decision boundary. Within each setting, we review the basic assumptions and classic results and present some recent advances in understanding neural network classifiers.

(1) In the smooth conditional probability setting, we establish fast—though not optimal—convergence rates for overparametrized ReLU neural networks (in the neural tangent kernel regime) trained with square loss. When the classes are separable with a positive margin, the misclassification rate improves to be exponentially fast and the resulting margin is lower bounded away from zero.

(2.1) In the smooth decision boundary setting, we shed light on the rate-suboptimal of existing neural network classification literature by investigating a novel localized version of the classical Tsybakov’s noise condition, under which statistical optimality can be attained utilizing the divide-and-conquer technique.

(2.2) We advocate studying the boundary complexity of classifiers and propose an explicit method for counting the boundary pieces of ReLU classifiers. Intriguingly, the boundary complexity behaves very differently than functional complexities and exhibits a negative correlation to classification robustness.

Here are the slides of the talk.