Approximation Rates for Shallow ReLU\(^k\) Neural Networks on Sobolev Spaces via the Radon Transform
Tong Mao, Jonathan W. Siegel, Jinchao Xu · SIAM Journal on Mathematical Analysis · 2026
Abstract. Let [Formula: see text] be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLU[Formula: see text] activation function can approximate functions from Sobolev spaces [Formula: see text] with error measured in the [Formula: see text]-norm. Utilizing the Radon transform and recent results from discrepancy theory, we provide a simple proof of nearly optimal approximation rates in a variety of cases, including when [Formula: see text], [Formula: see text], and [Formula: see text]. The rates we derive are optimal up to logarithmic factors, and significantly generalize existing results. An interesting consequence is that the adaptivity of shallow ReLU[Formula: see text] neural networks enables them to obtain optimal approximation rates for smoothness up to order [Formula: see text], even though they represent piecewise polynomials of fixed degree [Formula: see text].