<ul><li>Deep neural networks learn structured features from complex, non-Gaussian inputs, but the mechanisms behind this process remain poorly understood.</li><li>The first-layer filters learnt by deep convolutional neural networks from natural images resemble those learnt by independent component analysis (ICA).</li><li>FastICA, a popular ICA algorithm, requires a large number of samples to recover non-Gaussian directions from high dimensional inputs.</li><li>Vanilla online stochastic gradient descent (SGD) outperforms FastICA in feature learning.</li></ul>

Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions

Discover more