<ul><li>Researchers have developed fast algorithms and robust software for convex optimization of two-layer neural networks with ReLU activation functions.</li><li>The work leverages a convex reformulation of the weight-decay penalized training problem as a set of group-ℓ₁-regularized data-local models, utilizing polyhedral cone constraints.</li><li>In the case of zero-regularization, the problem is exactly equivalent to unconstrained optimization of a convex 'gated ReLU' network with non-singular gates.</li><li>The developed approaches outperform standard training heuristics and commercial interior-point solvers in terms of speed and performance.</li></ul>

Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions

Discover more