What is... an Equivariant Neural Network?
Lek‐Heng Lim, Bradley Nelson · Notices of the American Mathematical Society · 2023
We explain equivariant neural networks, a notion underlying breakthroughs in machine learning from deep convolutional neural networks for computer vision [KSH12] to AlphaFold 2 for protein structure prediction [JEP + 21], without assuming knowledge of equivariance or neural networks.The basic mathematical ideas are simple but are often obscured by engineering complications that come with practical realizations.We extract and focus on the mathematical aspects, and limit ourselves to a cursory treatment of the engineering issues at the end.We also include some materials with machine learning practitioners in mind.Let 𝕍 and 𝕎 be sets, and 𝑓 ∶ 𝕍 → 𝕎 a function.If a group 𝐺 acts on both 𝕍 and 𝕎, and this action commutes with the function 𝑓: 𝑓(𝑥 ⋅ 𝑣) = 𝑥 ⋅ 𝑓(𝑣) for all 𝑣 ∈ 𝕍, 𝑥 ∈ 𝐺, then we say that 𝑓 is 𝐺-equivariant.The special case where 𝐺 acts trivially on 𝕎 is called 𝐺-invariant.Linear equivariant maps are well-studied in representation theory and continuous equivariant maps are well-studied in topology.The novelty of equivariant neural networks is that they