<ul data-eligibleForWebStory="true"><li>A theoretical framework is proposed to analyze learning dynamics in deep neural networks using dynamical systems theory.</li><li>The framework introduces order-preserving and non-order-preserving transformations at the neuron level to redefine linearity and nonlinearity.</li><li>Different transformation modes lead to unique weight vector organization, information extraction, and learning phases.</li><li>Transitions between phases, including phenomena like grokking, can occur during training.</li><li>The concept of attraction basins in sample and weight spaces is introduced to characterize generalization and structural stability.</li><li>Metrics based on neuron transformation modes and attraction basins help analyze learning model performance.</li><li>Hyperparameters like depth, width, learning rate, and batch size influence these metrics for model optimization.</li></ul>

Network Dynamics-Based Framework for Understanding Deep Neural Networks

Discover more